Special Report from New Scientist Print Edition 27 March 03 Email traffic patterns can reveal ringleaders By looking for patterns in email traffic, a new technique can quickly identify online communities and the key people in them. The approach could mean terrorists or criminal gangs give themselves away, even if they are communicating in code or only discussing the weather. Finding communities in email "If the CIA or another intelligence agency has a lot of intercepted email from people suspected of being part of a criminal network, they could use the technique to figure out who the leaders of the network might be," says Joshua Tyler of Hewlett-Packard's labs in Palo Alto, California. At the very least, it would help them prioritise investigations, he says. Tyler and his colleagues Dennis Wilkinson and Bernardo Huberman, study email communication patterns and communities among networks of people. The trio wondered if they could identify distinct communities within Hewlett-Packard's research lab simply by analysing the IT manager's log of nearly 200,000 internal emails sent by 485 employees over a couple of months. They plotted the links between people who had exchanged at least 30 emails with each other, and found the plot included 1110 links between 367 people. In a network as large and complex as this, the plot alone will not tell you which groups people are. High "betweenness" So to pick them out, the researchers used a computer algorithm that looks for the critical links that form bridges between separate groups - what the team calls links with high "betweenness". By severing these links one by one, the algorithm gradually isolates people into different communities of groups who are emailing each other. To make sure the order in which links are severed does not distort the picture, the team repeated the task 50 times, each time cutting a different link first. Most individuals popped up in the same group every time; they were excluded from a group only if they failed to appear in it at least four times. The technique revealed 66 communities at the lab. And when the researchers compared the community members with the company organisation charts, they found that 49 of them contained people who all worked in the same department. In most of the others, the people were collaborating on a project. In a second investigation, the team plotted the same network of emails using a standard algorithm that, in effect, tries to arrange it in the least tangled way possible. This showed that the managers, including the director, tended to cluster in the middle. "This approach puts in the middle the people who have the most diverse range of contacts in the organisation - and these tend to be the leaders," says Tyler. In criminal investigations, detectives could look at global email traffic to see if known criminals are part of a wider community. The technique would also work for phone calls. Tyler admits that the way data is obtained and used could raise a raft of new questions over privacy. But at this early stage, the technique is a long way from being put into practice. Hazel Muir === Related Stories Word 'bursts' may reveal online trends http://www.newscientist.com/news/news.jsp?id=ns99993405 18 February 2003 Turing tests filter spam email http://www.newscientist.com/news/news.jsp?id=ns99993227 3 January 2003 Email to test 'six degrees of separation' http://www.newscientist.com/news/news.jsp?id=ns99991829 23 January 2002 Weblinks Information dynamics, HP Labs http://www.hpl.hp.com/shl/ ========== HURIDOCS-Tech listserv ========== Send mail intended for the list to <huridocs-tech@hrea.org>. Archives of the list can be found at: http://www.hrea.org/lists/huridocs-tech/markup/maillist.php To subscribe to the list, send a message to <majordomo@hrea.org>, with the following text in the message: subscribe huridocs-tech To unsubscribe from the list, send a message to <majordomo@hrea.org>, with the following text in the message: unsubscribe huridocs-tech If you have problems (un)subscribing, contact <owner-huridocs-tech@hrea.org>.
[Reply to this message] [Start a new topic] [Date Index] [Thread Index] [Author Index] [Subject Index] [List Home Page] [HREA Home Page]