Team Members:

Zuotian Li, Computer Graphics Technology, Purdue University, li4007@purdue.edu
Pengyu Ren, Art and Design, Purdue University, ren184@purdue.edu
Regan O’Connor, Computer Graphics Technology, Purdue University, oconno78@purdue.edu
Rachael Cai, High School, West Lafayette Junior Senior High School, cai.rachael@gmail.com
Justine Xue, High School, Ravenswood School for Girls, justine.xue03@gmail.com
Dr. Zhenyu Cheryl Qian, Art and Design, Purdue University, qianz@purdue.edu
Dr. Yingjie Victor Chen, Intelligent Visualization and Interaction Lab, Computer Graphics Technology, Purdue University, victorchen@purdue.edu   PRIMARY
Dr. Chen Guo, School of Media Arts & Design, James Madison University, guo4cx@jmu.edu   PRIMARY

Student Team: YES

Tools Used:

The backend: Python, NLTK, Gensim, Yellowbrick.
The front end was built upon HTML5, D3.js, pyLDAvis.js, Echart, Figma

Approximately how many hours were spent working on this submission in total?

200 hours

May we post your submission in the Visual Analytics Benchmark Repository after VAST Challenge 2019 is complete?YES

Video

https://framingvis.com/vast_mc3.mp4

---

Tasks and Questions:

1.Using visual analytics, characterize the different types of content in the dataset. What distinguishes meaningful event reports from typical chatter from junk or spam? Please limit your answer to 8 images and 500 words.

The different types of content in the dataset are the time of posting, geographic location, and content of posts shown in Fig. 1(a).

Before we did the t-SNE chart, we cleaned all the stopwords for each microblog message shown in Fig. 1(b), and used the TfidfVectorizer Library in Python to extract the vector representing the message keywords. If the message key content is meaningless without the stopwords, they are highly likely spam or junk.

Fig. 1(a) Original Data. Fig. 1(b) Cleaned stop words.

We used the LDA topic modeling and pyLDAvis.js to get an overview of 8 main topics during the time span (Fig. 2), for each of the topics, we had a glance of the most salient terms and the probability of each word to be belonging to a topic. Then we explored the details from the t-SNE cloud and other views to determine the events of microblogs.

Fig. 2 Topic modeling.

In the CloudAnnotator (Fig. 3), we can look at these different kinds of content. The t-SNE cloud overviews the clustering phenomena of the messages, while the keyword theme river and section views highlight the changes over time. The map view provides the geographical dimension of the posts’ locations. With these multiple views, the analyst of CloudAnnotator is able to detect, allocate, annotate, and respond to the major events taking place in the area through microblog messages in a timely manner. Taking advantage of these different forms of analysis means that spam can be efficiently ruled out.

Fig. 3 An overview of CloudAnnotator.

The multi-dimensional data is visualized in the t-SNE format to reveal clusters. For the clusters in the t-SNE cloud, if the message is similar or if repeating messages are shared often, we can easily identify the redundant content. Another example is when we read several messages in the cluster shown in Fig. 4 and enclose them in the rectangle, the word cloud corresponding to it shows most words uniformly, so the message in this cluster can be regarded as typical chatter.

Fig. 4 Use the t-SNE cloud and word cloud to identify spam.

To add context to the geographic location data, message locations are also transposed upon a geographic map of the microblog messages location below (Fig. 5).

Fig. 5 The geographic Map of microblog messages.

When the data points clustered in Fig. 3 are classified by type of content from keywords and graphed along the x-axis of time below, the theme river and pie chart help to differentiate meaningful events from less relevant chatter. Thus, in between the 06:30pm to 07:00pm, and 07:30pm to 08:00pm timeframes (Fig. 6), it is evident that meaningful events occur, as there is a noticeable spike of messages during these times.

Fig. 6 Use theme river to visualize theme changes over time.

Fig. 7 Use pie chart to display the distribution of keywords at a certain time period.

Fig. 7 shows pie charts which represent the microblog’s content and the number of posts during these spikes, at approximately 06:50pm and 07:50pm respectively. The data labels are summaries of the data points clustered in Fig. 3, wherein each relevant cluster has a specific label dependent on recurring keywords.

By compiling data points and referencing them together, it is possible to distinguish meaningless chatter from important events.

2.Use visual analytics to represent and evaluate how the level of the risk to the public evolves over the course of the evening. Consider the potential consequences of the situation and the number of people who could be affected. Please limit your answer to 10 images and 1000 words.

1) 5:00 - 5:30pm: POK Rally

Fig. 8 Cloud annotation of the beginning of the POK rally.
Fig. 9 Time and location of the beginning of the POK rally.

There is evidence that a POK Rally at the Abila City Park was taking place early in the evening from the messages seen in Fig. 8. The keywords ‘police’, ‘violence’ and ‘heavy’ were posted frequently, as seen in the pie chart from Fig. 9, in which all three words have prominent positions, implying a mild to moderate safety risk to the public.

Around 5:30pm we can see that the APD has closed the streets near the rally from the terms ‘apd’, ‘closed’, ‘streets’, and ‘surrounding’. The risk to the public remains unchanged. While a new dangerous keyword of ‘river_soldiers’ has appeared, it seems too limited in frequency to be a noteworthy risk to public safety.

Around 5:40 in the evening, there seems to be little to no risk to the public. A few microblog messages suggest that there was a subject stop on N. Berlau St / N. Taxiarchon Ave and a noise disturbance on 4777 N. Arkadou St, but these are not significant events.

2) 6:00 - 6:40pm: The Environmentalist

From the t-SNE annotation we found that there is a cluster discussing an environmental activity. The host of the event is Sylvia, and there are several speakers, like Dr. Newman, who is an environmental scientist, as annotated in Fig. 10. As the event is small and seems to be an informative talk, it presents no risk to public safety.

Fig. 10 6:00pm Environmental activities.

3) 6:40 - 7:00pm: Dancing Dolphin Fire

Fig. 11 Annotation of Dancing Dolphin Fire.
Fig. 12 Time and location of Dancing Dolphin Fire.

Around 06:45pm, we can observe a significant spike in the frequency of dangerous keywords ‘fire’ and ‘police’, as well as ‘dancing_dolphin’, which suggests that a fire has occurred at the ‘dancing dolphin’ location (Fig. 12). The relative frequency of the ‘fire’ keywords suggests a significant scale to the incident, and that it poses a moderate to severe threat to public safety. If the fire is spreading in a commercial district, it has the potential to hurt up to hundreds of people in the short term. In the medium to long term, repair costs to those the building owners could have a far more acute but limited impact, though this has minimal bearing on safety.

4) 7:00 - 7:20pm: Dancing Dolphin Fire Continued

Fig. 13 Fire continued at Dancing Dolphin.

Only 15 minutes later at 07:00pm, the frequency of the ‘fire’ keyword has significantly diminished, suggesting that the fire’s severity has been contained. Thus, at this moment the fire poses only a moderate risk to public safety.

5) 7:20 - 7:40pm: Hit and Run

Fig. 14 7:20pm Annotation of hit and run.
Fig. 15 07:20pm Hit and run cluster.

At 7:20pm a new keyword, ‘hit’, is introduced and quickly grows. By 7:40pm we can see new terms being used, like ‘pursuit’ and ‘van’ (Fig. 15). A micro-blog message claimed that there has been a hit and run by a black van. The event is a bicyclist was hit and uninjured at schaber, then police started to report the pursuit of the suspect car and released part of the license plate number. The car headed west, and the unfolding incident poses a moderate to severe threat to public safety.

6) 7:45 - 8:00pm: Shots Fired

Fig. 16 7:45pm Data on standoff

Around 07:45pm, there is a significant spike of dangerous keywords such as ‘police’, ‘apd’, ‘scene’, ‘shooting’’, ‘shots_fired’ and ‘report’, which suggest that a criminal incident involving gunfire has unfolded, and that there is a severe and pressing threat to public safety at the scene of the incident (Fig. 16). At this point in time, it seems likely that civilians may be caught in the crossfire of this dangerous incident, as well as potentially reckless driving from the van.

7) 8:00pm: The Standoff

At 08:00pm, the incident escalated in severity and danger, with the new keywords ‘scene’, ‘swat’, ‘hospital’ and ‘standoff’ being posted. The ‘hospital’ tag in particular suggests that some people have been injured, and given the ‘standoff’, this situation also seems likely to continue to be a severe threat to public safety.

8) 8:30pm: The Fire Continues

At 08:30pm, the ‘fire’ and ‘building’ tags have resurged in frequency, alongside the new tag ‘afdheroes’, suggesting that the previous fire has resurged, although the afd seems to have responded to the situation quickly. Thus, the fire crisis seems to have been managed, and poses only a mild to moderate risk to public safety. In terms of the unfolding criminal incident with the police, the new tags ‘life’ and ‘avoid’ seem to suggest that the public has started to actively avoid the scene of the incident, reducing the likelihood of civilian injuries. The consequences of this latter incident to public safety seems to pose a latent but still severe threat, as the ‘standoff’ continues.

9) 9:00 - 9:30pm: A Resolution

At 09:00pm, near the end of the evening, the ‘fire’ incident seems to have been managed, whilst the ‘abila’ rally and ‘police’ situation seem to remain pressing and severe threats to public safety. At the end of the evening, it is apparent that the multiple threats of a rally, traffic incident, fire and criminal incident have posed moderate and severe threats to the public. The rally seemed to be prominent, with heavy police presence around 5:30, but had no further news. The fire was also dangerous, resurging later and causing the top floor to collapse at around 20:33. The traffic incident where a bicyclist was struck by a car seemed to be unfolding but posed a minimal threat to the public as a whole. The most significant danger, however, was the unfolding criminal incident with the police and van, which had the potential for fatal consequences to the public and police, and was thus a severe threat to public safety.

A summary of the major events

Fig. 17 The major events recognized from CloudAnnotator.

3.If you were able to send a team of first responders to any single place, where would it be? Provide your rationale. How might your response be different if you had to respond to the events in real time rather than retrospectively? Please limit your answer to 8 images and 500 words.

Given the rationale of the previous response, the criminal incident has been evaluated to be the more pressing threat to public safety, and thus first responders should be sent to the location where it occurred. For example, at around 7:45, a shoot and standoff happened, and a police officer was shot and taken away by an ambulance, and was later reported not dead. We found the cluster pertaining to this event, and our Abila map shows the location of messages which reveals that it is highly possible that the person who published the microblog was there and the event happened around their location (Fig. 18).

Fig. 18 Location of the standoff event around 7:45.

From Fig. 18 we see that the shooting and stand-off happened around the General Grocer. When we hover the time window around 7:40 pm, the locations of all the geo-tagged messages are shown at the right bottom in Fig. 19, the standoff area should be near the shooting area which is along the Parla St. From the map, we can instantly identify the event, report to the nearby police and respond to it quickly.

Fig. 19 Location of event around 7:45.

Looking at Fig. 20, we can see that the relevant keywords of ‘pursuit’, ‘hit’ and ‘police’ are present in the former, but not present in the latter visualization. This is perhaps the difference between responding retrospectively and live based on the messages. In a live situation, there is incomplete and delayed information regarding the unfolding incidents, wherein you have to wait for a spike in the frequency of a particular keyword before you can confirm the situation. With the benefit of hindsight in responding retrospectively, you would have far more information on the breadth and development of these crucial situations, and likely be able to dispatch relevant police and firefighters to the locations before or when the incidents occur as a preventative measure to minimise consequences and their risk to public safety.

Fig. 20 The most frequently used keywords at 07:30pm and 07:40 pm (crisis point).

Furthermore, when responding retrospectively it is possible to discern the most dangerous event to the public relative to all the others which occur throughout the evening. This prioritisation would be far more difficult, and the designation of first responders may have been different if one was responding to the evening’s incidents live.

Moreover, when looking at Fig. 17, there is also the potential for real-time responders to be unable to parse through the multiple clusters of less relevant background information, such as chatter about POKrally and Kronosstar, in order to find more relevant information relative to public safety.

4. If you solved this mini-challenge in 2014, how did you approach it differently this year?

This year we adopted a modern machine learning method by using t-SNE and topic modeling algorithm. Additionally, we used a newer and more visually appealing method of visualizing the data drawn from t-SNE, from which the graph of microblog keywords plotted against time and pie chart of instantaneous message frequency was developed, in order to provide a clearer and more time-specific view of the situation.

Moreover, CloudAnnotator provides an annotation capacity to allow the analyst summarize clustered messages and identify events. The analyst can select multiple points in a cluster and summarize the events in a text input window. The manual annotation helps analysts categorize events and get work done faster.