From RPI: The Situations Room

RPI bloc

Rensselaer Polytechnic Institute

1

Times Union

6.1.17
bnearing@timesunion.com • 518-454-5094

2
Students at Rensselaer Polytechnic Institute demonstrate a new “situations room” that is connected the the Watson supercomputer from IBM.

2
Students at Rensselaer Polytechic Institute demonstrate the new “situations room” connected to the IBM supercomputer Watson.

3
Hui Su is the director of the Cognitive and Immersive Systems Lab at Rensselaer Polytechnic Institute.

Imagine being in a room to ask questions of one of the world’s most powerful computers. An artificial intelligence containing more information than the largest library, it can recognize you, hear you, see what you are pointing at, and even notice if you might be perplexed or inattentive. It knows all of your earlier work and might even anticipate your questions.

Such a room is now being created at Rensselaer Polytechnic Institute under a partnership with technology giant IBM and its supercomputer Watson, which first gained worldwide attention in 2011 when it beat humans in the TV show Jeopardy.

The so-called “situations room” is an example of what are called “cognitive and immersive systems,” in which the burgeoning field of artificial intelligence is melded with rapidly growing torrents of financial, health, education, and so-called “unstructured data” like social media posts spreading across an expanding constellation of networked computers, smartphones and other electronic devices.

Such rooms could be used by corporate board executives to develop more nimble acquisition strategies, at universities to teach students, for doctors considering diagnosis and treatment of patients, and by cybersecurity experts who want to stymie hackers, said Hui Su, director of the RPI Cognitive and Immersive Systems Laboratory.

There are only two such prototypes in the U.S. IBM is developing the other room in San Francisco to show potential customers. RPI launched the situation room project with IBM in November 2015; RPI was the first university to receive a modified Watson in 2013.

“We want the computer to be able to handle group discussion. To understand who is saying what and to deliver relevant information … even to anticipate where the discussion could be going,” said Su, during a recent demonstration.

Three large screens dominated the Watson room inside the Curtis R. Priem Experimental Media and Performing Arts Center, with one showing how the computer tracked people as they moved. The other two displayed data as Watson answered questions.

One demonstration showed how the system could be used to order food in a Chinese restaurant and was able to recognize when a student pointed to an item on a digital menu. The other showed how Watson could craft written narratives when asked questions about the Roman Empire or World War II.

A computer that has the power to process 800 million pages a second, Watson is the room’s brain, eyes, ears and voice. Since its Jeopardy victory, the computer has become much more powerful and compact, as IBM touts that the hardware necessary has shrunk from the size of an average living room to the size of three pizza boxes.

Watson is a “deep learning” software and hardware system that can reside in a computer, or in “the cloud” through a series of interconnected computers, according to an IBM spokeswoman. A key element of its operation is a process called “ingestion,” in which Watson collects data — some of it with no obvious connection — and then seeks to discern what patterns and connections might be there.

“Think of the room as an embodiment of Watson,” she said. “Watson is software that learns patterns. It is a form of machine learning,” she said.

So far, the situation room is rudimentary compared with human understanding, said Su.

For the technology to reach its full potential, Watson must master three distinct tasks:

Being able to recognize multiple people, hear and understand what each one says regardless of accent or intonation, comprehend questions, and even discern gestures, like pointing (and maybe even if someone has their eyes closed all the time).

Quickly sort through massive amounts of data to locate pertinent information, and be able to plan and anticipate where the discussion might lead.

Display data visually and create a narrative to answer questions in natural language that is easy to understand.

“Humans don’t stop to distinguish between the modalities they use to communicate. You point to something on the screen, move your hands and you talk about it, and I understand which parts are significant and interpret them,” Su said. “The first step to bridging that barrier is to make it possible for the machine to absorb that behavior in the correct order and understand which part is significant. They have to absorb and interpret multiple modalities simultaneously.”

Already, Watson is now being used by IBM for a growing variety of business applications. In Westchester County, it is helping doctors at Memorial Sloan Kettering review medical records and research to help diagnose and treat cancer patients.

On the strip in Las Vegas, it just started providing concierge services for guests at the Venetian Hotel on what to do, where to go and what to eat.

Last month, Watson analyzed video of the entire Masters’ golf championship, and created a highlight reel and spoken commentary based on recognition of such characteristics as level of crowd noise, gestures by players like fist bumps and high fives, and where shots landed on the course. Watson’s highlights were shown at the resort, although not broadcast. And in Los Angeles, the global advertising agency Saatchi used Watson to create thousands of customized Facebook advertisements for Toyota’s new $57,000 fuel cell passenger car, aiming each ad toward potential Facebook viewers based on 100 individual demographic characteristics that Watson had created.

BM also recently rolled out a free preview of what it calls the Watson Company Profiler, in which the system combines financial data from financial analysts Dun & Bradstreet, Standard & Poors Global Intelligence and Watson Discovery News. The service promises to help business executives “quickly identify your next acquisition target, emerging competitor or sales lead.”

See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

RPI campus

Founded in 1824, Rensselaer Polytechnic Institute is the nation’s oldest technological research university.

The university offers degrees from five schools: Engineering; Science; Architecture; Humanities, Arts, and Social Sciences; and the Lally School of Management; as well as an interdisciplinary degree in Information Technology and Web Science.

Institute programs serve undergraduates, graduate students, and working professionals around the world. Nearly 29 percent of undergraduate students this year are from areas outside of the Northeast. First-year students hail from 43 states, in addition to the District of Columbia and Puerto Rico, and from countries all around the world.
Rensselaer offers more than 145 programs at the bachelor’s, master’s, and doctoral levels. Students are encouraged to work in interdisciplinary programs that allow them to combine scholarly work from several departments or schools. The university provides rigorous, engaging, interactive learning environments and campus-wide opportunities for leadership, collaboration, and creativity.

For almost two centuries, Rensselaer has maintained its reputation for providing an undergraduate education of undisputed intellectual rigor based on educational innovation in the laboratory, classroom, and studio.

Driven by talented, dedicated, and forward-thinking faculty, Rensselaer has dramatically expanded the research enterprise by leveraging our existing strengths and focusing on five signature research areas: biotechnology and the life sciences; energy and the environment; computational science and engineering; nanotechnology and advanced materials; and media, arts, science, and technology.

The Institute is especially well-known for its success in the transfer of technology from the laboratory to the marketplace so that new discoveries and inventions benefit human life, protect the environment, and strengthen economic development.

Advertisements