Comments: Tried commenting on Brandon's post but it wouldn't let me.... so I posted on Jill's
Summary:
This paper is about watching the way analysts work and analyzing how they think in order to create tools that will help them do their job more efficiently and possibly faster. There were five analysts that took part in the two scenarios that PNNL created for this observational study. The analysts jobs were a wide range if things such as cyber-security, threat analysis, critical infrastructure protection, counterintelligence, and nuclear non-proliferation. Each scenario was fabricated so none of them were actual cases needed to be solved, and none of them had a "correct solution". For each scenario the analysts were given typical analysis tools that they normally would have if they were a real case.
Scenario 1 consisted of a domestic terrorism situation. Each analyst was asked to play the role of an FBI agent and were given "evidence" from the case that were collected by "another specialist". They all had electronic copies of notes, floor plans, background information of the group supposedly responsible for the attack, and a list of online associates of the group. Each analyst was to work alone and piece together all of the evidence so that they could have a verdict. They were each given two weeks and were able to use tools available to them in order to figure out the case. They were also asked to go about their usual ways in solving something, this way PNNL would get a better idea of how analysts think and would be getting feedback in order to create tools later on. After the two weeks were over, PNNL discovered that many of the analysts relied on basic application tools instead of using tools that were provided to them. They developed some strategies, and pieced together evidence in the ways that they were used to. One strategy was called "competing hypothesis" where the analyst would come up with some hypotheses on what took place, and figured out which evidence would support each hypothesis. The one with the most evidence was likely to be the most credible, so they would run with it and piece together more evidence with certain situations.
There were different ways the analysts collected information. One analyst printed all of the evidence, spread it around her, and then started making piles by what type of evidence they were (emails, interviews, phone logs). Within each group she organized them more and put them into subtopics. Then she drew a graph in PowerPoint in order to visually see all of the evidence in front of her.
Another analyst made five hardcopies of each piece of evidence and put them into different folders that represented different types of relationships. Then she did the same thing on her computer (placing files in folders). A third analyst did something called "triage" which means ordering the evidence by relevance and importance. From this scenario PNNL learned that computer tools such as spreadsheets, drawing software, indexing and linking of information would help the analysts by reducing the time it would take to manually print and organize all of the evidence.
Scenario 2 deal with an investigative team where each analyst was asked to play the role of a detective, while one would be a case chief who would oversee the work that each detective has done. The "Gregorian Brotherhood" was the gang would was responsible for many of the crimes that the team was investigating. Each investigative team was supposed to find patterns amongst them members across the multiple districts that they crimes occurred. Some analysts used common and everyday tools in order to show data from different perspectives. Microsoft Excel was one tool that was used my multiple analysts in order to visually represent the evidence.
Another analyst tore out calendar from their planner and circled dates in which the crimes occured. A third analyst actually highlighted crimes in different colors based on the type of crime that it was. Analysts had to also figure out which evidence was credible and which may have been fabricated or "spoofed". Some evidence may have even been ambiguous or incomplete. For each analyst to figure out what evidence is credible will greatly depend on what kind of background they have.
With the findings from each scenario the authors tried to understand how analysts think and what kind of tools may benefit them. They said that "a computer system that could generate a set of standard views from a data set would provide the analyst a way to systematically explore and investigate the data along specific themes or patterns" (19). Since many of the analysts used basic tools such as pens and highlighters, it would be beneficial to incorporate the ease of use of these devices to allow for encoding and annotating information.
Discussion:
I found this paper to be rather interesting. When I first read the title and saw that it was 10 pages I thought to myself, 'great another 10 page paper that I'm probably not going to understand or be interested in". The scenarios and actually seeing how each analyst worked in order to figure out what patterns there were in the data was what kept me interested. I think there could be a lot of future work that would involve creating computer tools that would allow an analyst to be comfortable with doing everything digitally instead of resorting to other methods that they are most familiar with. A lot of the analysts actually made hard-copies of everything, and I myself do this on occasion because I am more familiar with having a copy in my hand and studying it, then reading something on the computer and studying it in that way. I think there could be software implementations that allow for much easier graph drawings, the ability to highlight information (i.e. actually looking like a highlighter over text vs. the backgrounds you have to do in word in order to see that it is a different color). I know there are some tools that are already like this, but one full package would be a good start so that it is easily obtainable by organizations for their analysts to use.
No comments:
Post a Comment