Entry Name: "SDU-Wang-MC2 "
VAST Challenge 2020
Mini-Challenge 2
Team Members:
Yue Zhao, Shandong University, jack.zhao9802@gmail.com
Xuan Zhang, Shandong University, xwxztq@gmail.com
Chuyang Zhao, Shandong University, 2741591545@qq.com
Yunhai Wang, Shandong University, cloudseawang@gmail.com PRIMARY
Student
Team: NO
Tools Used:
Python
Vue.JS
Vega
D3.JS
Git
GitHub
VS Code
Approximately how many hours were spent working on
this submission in total?
50 hours.
May we post your submission in the Visual Analytics
Benchmark Repository after VAST Challenge 2020 is complete? YES
Video
Questions
Given
the images and text files as well as machine learning outputs, use visual
analytics to answer the questions below to improve and understand machine
learning outputs and track provenance, uncertainty, and confidence in machine
learning results. Ultimately, you must link multiple data types to identify the
group CGCS is seeking.
1 Examine the outputs from the model either from the
detection results provided or the results from a model you chose. Which objects
were identified well by the model and which were not? Please limit your answer
to 5 images and 250 words.
Answer:
The histogram we designed can display the classification results
of the model from the dimensions of People and Object. In the People dimension,
each histogram will show the distribution of confidence in all classification
results for all pictures of each person. In the dimension of Object, the
results of each Object classification are aggregated together and displayed in
the histogram according to the confidence distribution.
For example, as shown in Figure 1, the classification result
of cupcake-paper is relatively poor, and the distribution of confidence is in a
relatively low range. The result of hair-clip is
better, and the confidence of the results is relatively high as shown in Figure
2.
Figure 1
Figure 2
2 Demonstrate your process for using visual analytics to correct
for classification errors in the results. How do you represent confidence and
uncertainty? How could the correction process be made more efficient? Please
limit your answer to 10 images and 500 words.
Answer:
Our visual analysis system can directly correct the
classification results of the model in the Image Cloud. Right-click on the
corresponding picture, and the pop-up menu displays the detailed information of
the corresponding picture, including confidence and classification results, and
to whom this picture belongs.
Figure
3
The classification result can be corrected by clicking the
button named Correct the Classification. Fill in your correction results
in the dialog box that pops up afterwards. Then the corresponding picture will
be marked in red in the Image Cloud to indicate that it has been successfully
corrected.
Figure
4
3 Characterize
the distribution of objects across the forty people.
a.
Which
people have which objects? Please
limit your answer to 8 images and 250 words
Answer:
In the parallel coordinate system, we can clearly and
conveniently view the items owned by each person or even the items owned by
multiple people. For example, when we want to see which objects are in Person
21, we only need to hover the mouse over the svg
element corresponding to Person 21 to see what the corresponding objects are.
Figure
5
b.
Identify
groups of people that have object(s) in common. Please limit your answer to 10
images and 500 words.
Answer:
In order to find people with common items, we need to find
the items owned by each person through the parallel coordinate system first,
and then observe the distribution to find people similar to the distribution of
the items they own, and confirm our conjecture by observing the distribution of
the parallel coordinate system. Because some items are owned by most people,
and some items are classified incorrectly, it is not appropriate to directly
use the program to calculate the group of people with the common items. Using
our system, we can observe the distribution of items owned by everyone At the
same time, observe the confidence scores distribution of the image cloud and
the items to know which items may have classification errors and which items
are generally owned by most people. Based on this, we can find people with
similar distribution of items more accurately.
As shown in Figure 6, in Image Cloud, by adjusting the
People/Object dimension and the two-level image cloud, we can intuitively
understand which objects are shared by people through pictures.
Figure 6
At the same time, in conjunction with Parallel Coordinate, we
handle more complex relationships. As shown in Figure 7, our system can also
see what objects are available to multiple people. For the object, vice versa.
This function can facilitate us to find totems and complete mainline tasks.
Figure
7
4 Which group do you
think is the most likely group with the totem? What is your rationale for
that assessment? Please limit your response to 5 images and 300 words.
Answer:
The totem we
found is PaperPlate. By looking at the image
cloud generated by our visualization system, we can find that the following
items only belong to a small group of people, namely paperplate,
bluesunglasses, yellowBalloon,
noisemaker, vancouver card. The YOLO model is
not very ideal for the classification of some items. We use the algorithm
results combined with manual correction
to get the person group of the above candidates: paperPlate
(7, 13, 19, 25, 31, 37), BlueSunGlasses ( 2,
7, 9, 13, 17, 27, 39), yellowBalloon (1, 2, 3,
4, 5, 6), noiseMaker (1, 2, 12, 15, 20, 36), vancouverCard (1, 26, 39).
Figure
8
According to the results of our image cloud,
there are too many people containing BlueSunGlasses,
and this item is not special, so the BlueSunGlasses
may not be totem; there are only 1, 3, 4 people with yellowBalloon,
and the item is not uncommon. So the possibility is also low; the number of
people with noisemaker is only 37, so noisemaker is also excluded; the number
of people with vancouverCard is also small,
and the items contained in each are very different, so the item is also
excluded. The people with PaperPlate include
7, 13, 15, 19, 27, 37, etc., which is in line with the nature of small groups,
and the pattern of the item is not common. It is unlikely that several people
have the same pattern paper plate, so we think that this item is totem.
5 Process question:
Did you choose to use the object recognition model results provided or use your
own machine learning algorithm? Why did you make that choice? What was the
biggest challenge you faced? Please limit your response to 3 images and 300 words.
Answer:
Our final choice
is the official training set results. Because this is to ensure the robustness
of our system. Without requiring perfectly accurate results, our system still
needs to be able to work effectively, which is also the principle advocated by
software engineering for a long time. However, during development practice, the
poor classification results still bring us huge challenges. This makes us have to carry out a lot of manual intervention and correction
to smoothly analyze the task.
Figure
9