Entry Name:  "UHasselt-Aerts-MC2"

VAST Challenge 2021
Mini-Challenge 2

 

 

Team Members:

Jannes Peeters, UHasselt, jannes.peeters@uhasselt.be, JOINT PRIMARY

Jelmer Bot, UHasselt, jelmer.bot@uhasslet.be, JOINT PRIMARY

Danai Kafetzaki, UHasselt – KU Leuven, danai.kafetzaki@uhasselt.be

Jan Aerts, UHasselt, jan.aerts@uhasselt.be



Student Team:  No

 

Tools Used:

·         Tableau for exploration.

·         R-studio for data analysis.

·         Python with numpy and pandas and Ruby to pre-process data.

·         ArangoDB as backend for the visualizations and for matching logic (https://www.arangodb.com/).

·         Svelte and D3 for custom visualizations.

 

 

Approximately how many hours were spent working on this submission in total?

300 hours in 7 weeks by 4 people.

 

May we post your submission in the Visual Analytics Benchmark Repository after VAST Challenge 2021 is complete? Yes

 

Video

https://youtu.be/GJCdzg80qYE

 

Links to the interfaces shown in the video*:

* The tools mentioned below use mixed http + https content. Please see our blogpost for instructions on how to access the interfaces using chrome, as well as higher-resolution images.

 

·         Credit card to loyalty card matching:

o   https://vast2021-cc2ly.vercel.app/

o   https://vast2021-corrmatrix.vercel.app/

·         GPS activity & Points of Interest:

o   https://vast2021-gps.vercel.app/

·         POI annotation & car to credit card matching:

o   https://vast2021-gps2cc.vercel.app/

·         Exploring meetings:

o   https://vast2021-meetings.vercel.app/

 

 

 

 

As a visual analytics expert assisting law enforcement, your mission is to identify which GASTech employees made which purchases and identify suspicious patterns of behavior. You must cope with uncertainties that result from missing, conflicting, and imperfect data to make recommendations for further investigation.

Use visual analytics to analyze the available data and develop responses to the questions below. In addition, prepare a video that shows how you used visual analytics to solve this challenge. Submission instructions are available here. Entry forms are available for download below.

Questions

1 –– Using just the credit and loyalty card data, identify the most popular locations, and when they are popular. What anomalies do you see? What corrections would you recommend to correct these anomalies? Please limit your answer to 8 images and 300 words.

 

Our initial exploration used the number of transactions as indication of popularity. The most popular locations are shown in Figure 1. Most locations have more credit card transactions than loyalty card transactions, except for Kalami Kafenion.

Figure 1 . The number of transactions per location

Visualizing the time of a transaction revealed distinct popularity patterns (Figure 2). We found 6 location groups as indicated in the color legend. Interestingly, most general stores are visited only in the afternoon and evening. We found two anomalies: for some locations all transactions are at 12:00, and the Kronos Mart transaction times are at unexpected hours.

Figure 2 . Transactions over time. Locations were grouped manually.

We used two metrics to match credit cards with loyalty cards, as shown in Figure 3. Figure 5 shows the resulting matches.

 

Figure 3 . The credit-to-loyalty card matching process.

 

Figure 4 . Credit-to-loyalty card price vector correlations.

 

Figure 5 . Relations between credit and loyalty cards.

We found several anomalies: Loyalty card transactions at the Kronos Mart occur one day earlier than the corresponding credit card transactions. Loyalty and credit card transactions may differ in price (Figure 6), we speculate people are paying for cash. Figures 7 and 8 show the non-1-to-1 matches.

Figure 6 . Price differences between credit and loyalty card transactions. Darker colors indicate more transactions.

 

Figure 7 . L6267 is used by both 6899 and 6691. On Saturday 11, there is a 10-minute gap between transactions at different locations.

 

Figure 8 . L3288 is used by 1286 and 9241. 1286 uses L3288 for lunches of around 50 Kr.

 

2 – Add the vehicle data to your analysis of the credit and loyalty card data. How does your assessment of the anomalies in question 1 change based on this new data? What discrepancies between vehicle, credit, and loyalty card data do you find? Please limit your answer to 8 images and 500 words.

 

We estimated missing transaction times after matching cars to credit cards and locations to GPS coordinates (see Question 3 for details). Specifically, transaction times were set to 1 minute before departure at the location. Figure 12 provides an example of this process. Figure 9 shows the resulting time-distributions for the breakfast locations. Loyalty card transaction times of transactions without matching credit card transaction were similarly estimated. The remaining unknown transaction times could potentially be estimated manually by looking at reoccurring transaction patterns when the matching interface (Figure 12) supports dragging transactions to another time.

 

Figure 9 . Estimated transaction times.

We tried to explain the non-1-to-1 credit card-to-loyalty card matches (from Figure 5) based on the car-to-credit card matching (Question 3). Figure 10 shows the stops of car 5 (Isak Baza) colored by the type of location (as identified in Question 3) with the transactions of credit cards 6899 and 6691. The transactions of 6899 match with car 5, shown by the transaction’s circle overlapping with the stop’s rectangle. The transactions of 6691 did not match any car and are shown above the stop rectangles. We found no clear reason why both credit cards use the same loyalty card.

 

Figure 10 . Stops of car 5 (colored bars) with transactions of credit cards 6691 and 6899. Transactions centered on a stop match in location and time. Transactions above the stops could not be matched to the stop. Tooltips provide details and give access to the (un)match button. The `match all` button accepts all shown possible matches.

 

Figure 11 shows the matching cars for credit cards 1286 and 9241. It appears that these people (Adra Nubarron (22) and Isande Borrasca (28)) regularly eat lunch together and alternate who drives. In addition, Figure 8 turns out to be quite misleading, as the transactions at 12:00 of credit card 9241 actually occurred in the morning, obscuring the explanation of a shared lunch.

 

Figure 11 . Stops of cars 22 and 28 with transactions of 1286 and 9241. The pink stop indicates a period of missing GPS signal. Places visited together are annotated.

 

Through matching the cars to credit cards, we also discovered that the Kronos Mart transaction times were still incorrect (Figure 12). On average the transactions occurred 11 hours and 54 minutes after departure, with a standard deviation of 4 minutes and 15 seconds. Consequently, it seems that the loyalty card transaction date was correct and the credit card transaction times are wrongly recorded.

 

Figure 12 . An example of incorrect Kronos Mart transaction time.

 

3 – Can you infer the owners of each credit card and loyalty card? What is your evidence? Where are there uncertainties in your method? Where are there uncertainties in the data? Please limit your answer to 8 images and 500 words.

 

First, we detected stationary GPS periods. We used a simple decision tree to classify the periods between subsequent GPS measures as moving, stationary, or missing (Figure 13). Figure 14 was used to determine the thresholds in the decision tree. Note, car 28’s GPS is offset by a fixed translation.

 

Figure 13 . Decision tree for GPS classification.

 

Figure 14 . Begin and end GPS positions of non-moving periods. Red lines indicate missing periods. Black arrows indicate the offset of car 28.

 

Especially car 9 has many periods with missing GPS data (Figure 15). Figure 16 demonstrates how missing periods could be filled in. We ignored periods of missing data in the remainder of this analysis.

 

Figure 15 . Timeline of GPS activity for cars 1, 9, 28 and 107. Red indicates missing periods, black spikes indicate movement.

 

Figure 16 . Filling in periods of missing GPS data (animated gif). Transaction times could be used to estimate the deprature time in the missing period..

In Figure 13, by clicking a single GPS measure (arrival or departure), all other arrival and departure locations within a configurable distance were selected and the median selected GPS coordinates were stored as Point of Interest (POI). For each POI, we annotated a type and name based on the tourist-map, a busyness trace, histograms of arrivals and departures and durations (Figure 17). The busyness trace shows how many cars were present over a day at that location, rescaled between 0 and 1. Homes have a U-shaped busyness trace.

 

Figure 17 . Points of Interest, coloured by type and sized by the number of distinct visiting cars.

 

Car-to-credit card and POI-to-location matches were found in an iterative interactive approach shown in Figure 18. We used two approaches: one based on correlations and one based on classification.  The resulting matches are shown in Figure 19.

 

Figure 18 . The car-to-credit card matching process.

Figure 19 . Car-to-credit card matches.

 

Most cars are matched to one credit card. However, the trucks regularly change driver on different (parts of the) days. Car 29 seems to have changed owner, as both the credit card and loyalty card changed in the second week (Figure 20). We did not consider potential implications of this in our analysis.

 

Figure 20 . Matched credit cards of car 29.

 

The evidence of our method is grounded in transactions of a credit card that occur in time periods where a car is stationary at the POI belonging to the transactions’ location. Initially there is uncertainty in the method, as it produces many false-positives because there are no POI-to-location constraints yet. However, as these constraints are added, the picture becomes clearer. The method is sensitive to uncertainty in the data. Specifically, incorrect transaction times (especially those at 12:00) produce many incorrect (low rated) matches.

 

4 –– Given the data sources provided, identify potential informal or unofficial relationships among GASTech personnel. Provide evidence for these relationships. Please limit your response to 8 images and 500 words.

 

Adra Nubarron (22) and Isande Borrasca (28) regularly meet each other (Figure 11).

 

We found several households among the GASTech personnel (using Figures 17 and 20):

·         Linnea Bergen (6), Kanon Herrero (25), Bertrand Ovan (29)

·         Inga Ferro (13), Loreto Bodrogi (15), Isia Vann (16), Hennie Osvaldo (21)

·         Lidelse Dedos (14), Birgitta Frente (18)

·         Sven Flecha (17), Minke Mies (24), Brand Tempestad (33)

·         Adra Nubarron (22), Varja Lagos (23) Felix Resumir (30)

 

There was a (surprise) party for Lars Azada on Friday 10 (Figure 21). Lars was one of the last to arrive at his own house. Vira Frente went to buy something at Alberts Fine Clothing before Lars arrived.

 

Figure 21 . Surprise party of Lars Azada (2). Circles are an Upset visualization indicating which cars were present, color indicates households, black for 1-person households.

 

There is a regular pattern of security personnel guarding executives’ homes at night (Figure 22). Two perimeter / site control employees are present in sequence between midnight and 7AM. We expect some kind of guard duty. We find it suspicious that the next day this duty would appear (Thu 16th) there is more-than-usual truck activity in the afternoon.

 

Figure 22 . Nightly guard-duty pattern of perimeter control personnel at executives' homes.

 

Elsa Orilla (7) and Brand Tempestad (33) (both drill technicians) regularly meet around lunch for 2 hours at the Chostus Hotel (Figure 23). Until the morning of Friday 17, car 31 is parked at the hotel without its driver being present.

 

Figure 23 . Elsa Orilla (7) and Brand Tempestad (33) regularly meet around lunch time for 2 hours at the Chostus Hotel.

 

Hennie Osvaldo (21) does not stay at the same house every night. On Wednesday nights and Sundays, as well as every day around 6PM, he visits the household of Lidelse Dedos (14) and Birgitta Frente (18). We expect he is in a relation with Birgitta, as he is never alone at the house with Lidelse for more than half an hour (Figure 24).

 

 

Figure 24 . Hennie Osvaldo (21) regularly visits the household of Lidelse Dedos (14) and Birgitta Frentet (18) on Wednesday nights and Sundays.

 

Nils Calixto (1) and Isak Baza (5) met 7 times in Ouzeri Elian. Their arrival, departure and transaction times are consistently quite similar. Isande Borrasca (28) often there as well, but arrives and departs at different times (Figure 25).

 

 

Figure 25 . Nils Calixto (1) and Isak Baza (5) are regulars at Ouzeri Elian.

 

5 –– Do you see evidence of suspicious activity? Identify 1- 10 locations where you believe the suspicious activity is occurring, and why    Please limit your response to 10 images and 500 words.

 

Figure 26 shows the price-distribution per location and four suspicious transactions:

·         Nils Calixto (1) spent 10.000Kr at Frydos Autosupply n’ More on Monday 13. There are 3 other transactions by his card that day which do not match his car’s location. Is another credit card with the same last four numbers used? Did Nils make these transactions without his car? Why would he need another car?

·         Axel Calzas (11) spent 1.239Kr at Albert’s Fine Clothing on Friday 17. We found no meeting to explain this purchase. However, Axel was at Hippokampos while three executives were there as well on Sunday 19. Did Axel come into money suddenly?

·         Sten Sanjorge Jr. (CEO) (31) spent 600Kr at Chostus Hotel on Saturday 18. We expect he arrived on Kronos Friday 17 and stayed at the hotel at least until Monday 20.

·         Bertrand Ovan (29) spent 478Kr at General Grocer on Saturday 18. Did he buy weekly groceries for his household?

  

Figure 26 . Price-distributions per location.

 

There was a transaction at Carlyle Chemical Inc (Tuesday 7, truck 105) with a 960Kr price difference (Figure 6). Is it related to the activity on Thursday 16? Most of the trucks repeatedly drive a route without stopping (Figures 27-30). Is this related to the arrival of the CEO on Friday 17?

 

Figure 27 . Activity of car 101 on Thursday 16 (animated gif). Repeated route: Stewart and Sons -> Nationwide Refinery -> Office -> Airport.

 

Figure 28 . Activity of car 104 on Thursday 16 (animated gif). Repeated route: Office -> Airport.

Figure 29 . Activity of car 105 on Thursday 16 (animated gif). Repeating route: Office -> Carlyle Chemical Inc. -> Office -> Katerina's Café.

 

Figure 30 . Activity of car 106 on Thursday 16 (animated gif). Repeating route: Scrapyard -> Stewart and Sons Fabrication -> Office -> Airport.

Nils Calixto (IT helpdesk) (1) goes to the office at night on multiple occasions ( Figure 31 ).

 

  

Figure 31 . Nils Calixto (1) goes to the office at night on multiple occasions.

 

 

Figure 32 shows all meetings with two executives present. They live close by Hippokampos. They meet for golf twice (Orhan Strum (32) and Sten Sanjorge Jr. (31) only the second time). Sten meets Ada Campo-Corrente (10) and Ingrid Barranco (4) for lunch on Friday 17 and Saturday 18, respectively, but never eats with any of the other executives. Ada and Willem Vaco-Pais (35) have lunch together at the same time on Saturday 18, but not at the same place.

 

Figure 32 . Meetings with two executives present.

 

For dinner on Sunday 19, CEO Sten Sanjorge Jr. ate at Guy’s Gyros with security personnel (Figure 32). Ingrid Barranco (4) went home. The other executives ate at Hippocampos with engineering staff (Figure 34). In both restaurants, most other employees present seem to be regulars.

 

Figure 33 . CEO Sten Sanjorge Jr. (31) eats with security personnel at Guy's Gyros on Sunday 19.

 

Figure 34 . Ada Campo-Corrente (10), Orhan Strum (32), and Willem Vaco-Pais (35) eat with engineering personnel at Hippokampos.

6 –– If you solved this mini-challenge in 2014, how did you approach it differently this year?

Jan Aerts was involved in a submission to the VAST challenge in 2014. The current submission is however unrelated and does not use approaches, tools or knowledge gained from the 2014 submission.