Google added a great new service, Dashboard, that summarizes data stored for a Google account — see MY ACCOUNT>PERSONAL SETTINGS>DASHBOARD.

“Designed to be simple and useful, the Dashboard summarizes data for each product that you use (when signed in to your account) and provides you direct links to control your personal settings. Today, the Dashboard covers more than 20 products and services, including Gmail, Calendar, Docs, Web History, Orkut, YouTube, Picasa, Talk, Reader, Alerts, Latitude and many more. The scale and level of detail of the Dashboard is unprecedented, and we’re delighted to be the first Internet company to offer this — and we hope it will become the standard.”

This is a good move on Google’s part. But while there’s a lot of information included, it’s not everything that Google knows about you — e.g., data in cookies, click throughs data from search results and information from companies it’s acquired, like Doublclick. Still, it is a big step in a positive direction.

Yesterday was the first time a truly voter verifiable voting system was used in any binding government election, thanks in part to work being carried out at UMBC’s Cyber Defense Lab under the direction of Alan Sherman.

Takoma Park, MD used the Scantegrity system for its municipal election after testing it in a mock election last April. Technology Review has a story, First Test for Election Cryptography, that quotes Anne Sergeant, the chair of the Takoma Park board of elections

“Before trying Scantegrity in an official election, the city held a mock vote in April to work out kinks in the system. In that test, she says, about 30 percent of participants went home and used the system to verify their votes. Sergeant says that Scantegrity representatives talked extensively with voters and election officials after the April test and have improved their system accordingly. “I hope we can provide an experience where people walk away and say, ‘That was awesome,’” she says. “It’s a goal to which we aspire.”

The Scantegrity system was created by a group of universities, including UMBC. A voter uses a paper ballot marked with invisible ink, which is exposed with a special marker. That marker reveals a code, which the voter can then use to check online whether their vote was tabulated correctly.

Ben Adida has been auditing the election and documenting the process on his blog.

See also the ComputerWorld story, E-voting system lets voters verify their ballots are counted, and audio report on WAMU.

DARPA will hold the DARPA Network Challenge to explore how “broad-scope problems can be solved using Internet-based technologies.

“To mark the 40th anniversary of the Internet, DARPA has announced the DARPA Network Challenge, a competition that will explore the role the Internet and social networking plays in the timely communication, wide area team-building and urgent mobilization required to solve broad scope, time-critical problems.

The challenge is to be the first to submit the locations of ten moored, 8 foot, red weather balloons located at ten fixed locations in the continental United States. Balloons will be in readily accessible locations and visible from nearby roadways.”

According to the rules, the balloons will be on display from 10:00AM to 4:00PM on Saturday, 5 December 2009. A prize of $40,000 will be awarded to the first participant to submit the latitude and longitude of all ten weather balloons within the contest period, which ends on 14 December 2009.

Golden Balls is a UK game show with a final round, Split or Steal, that is similar to the prisoner’s dilemma. The two contestants have to simultaneously choose to split the prize or try to steal it. If both choose split, they each get half. If one chooses split and the other steal, than the stealer gets it all. If they both choose steal, neither gets anything. While the payoff matrix is not exactly that for the PD, it has a similar effect on the strategy. Check out this video of a Split or Steal round for £100,000. (Spotted on Hacker News)

In the Fall of 2007, two MIT students carried out a class project exploring how presumably private data could be inferred from an online social networking system. Their experiment was to predict the sexual orientation of Facebook users who make their basic information public by analyzing friendship associations. As reported in the Boston Globe last month, the students’ had not yet published their results.

Well, now they have — in the October issue of the First Monday, “one of the first openly accessible, peer–reviewed journals on the Internet”.

Carter Jernigan and Behram F.T. Mistree, Gaydar: Facebook friendships expose sexual orientation, First Monday, v14, n10, October 2009.

The paper has a lot of detail on the methodology for collecting the data and how it was analyzed. Here’s the abstract.

“Public information about one’s coworkers, friends, family, and acquaintances, as well as one’s associations with them, implicitly reveals private information. Social networking Web sites, e–mail, instant messaging, telephone, and VoIP are all technologies steeped in network data — data relating one person to another. Network data shifts the locus of information control away from individuals, as the individual’s traditional and absolute discretion is replaced by that of his social network. Our research demonstrates a method for accurately predicting the sexual orientation of Facebook users by analyzing friendship associations. After analyzing 4,080 Facebook profiles from the MIT network, we determined that the percentage of a given user’s friends who self–identify as gay male is strongly correlated with the sexual orientation of that user, and we developed a logistic regression classifier with strong predictive power. Although we studied Facebook friendship ties, network data is pervasive in the broader context of computer–mediated communication, raising significant privacy issues for communication technologies to which there are no neat solutions.”

As we had previously noted, this datamining exercise only accesses information that Facebook users explicitly choose to make public. The authors note that their analysis “relies on public self–identification of same–gender interest in Facebook profiles as a sentinel value for LGB identity”. The privacy vulnerability is that the default setting for a Facebook account is that friendship relations are public and you can not control the privacy settings of your friends. So if your leave your friend list public and many of your Facebook friends open up their profiles, it may be possible to draw reasonable inferences about your age, gender, political leanings, sexual preferences and other attributes.

In next week’s ebiquity meeting (10:15 EDT Tue 10/6), Lance Byrd and Set Cruz will talk about Blackbook, a graph analytic processing platform for semantic web data.

Blackbook3 is an RDF middleware framework for integrating data and executing algorithms that relies on open standards and “best-of-breed” open source technologies, including Jena, Lucene, JAAS, D2RQ, Hadoop, HBase and Solr. Blackbook3 has a plug-and-play, loosely–coupled architecture, supports SOAP and REST interfaces, offers SPARQL and linked data endpoints and can run in environments where high confidentiality is required.

The talk will discuss the current and future use cases for Blackbook3 as well as broader knowledge discovery and dissemination issues for RDF applications. You can participate remotely via dimdim starting at 10:15 EDT October 6.

David Easley and Jon Kleinberg have made available a free pre-publication draft of a new book, Networks, Crowds, and Markets: Reasoning About a Highly Connected World, to be published by Cambridge University Press in 2010. The book is based on an inter-disciplinary undergraduate course, Networks, that they teach at Cornell.

They say about their book

“Over the past decade there has been a growing public fascination with the complex “connectedness” of modern society. This connectedness is found in many incarnations: in the rapid growth of the Internet and the Web, in the ease with which global communication now takes place, and in the ability of news and information as well as epidemics and financial crises to spread around the world with surprising speed and intensity. These are phenomena that involve networks, incentives, and the aggregate behavior of groups of people; they are based on the links that connect us and the ways in which each of our decisions can have subtle consequences for the outcomes of everyone else.
    Networks, Crowds, and Markets combines different scientific perspectives in its approach to understanding networks and behavior. Drawing on ideas from economics, sociology, computing and information science, and applied mathematics, it describes the emerging field of study that is growing at the interface of all these areas, addressing fundamental questions about how the social, economic, and technological worlds are connected.”

Download the 828-page (!) draft of Networks, Crowds, and Markets in pdf here.

The New York Times reports that the data for the Netflix Prize 2 will include more information about the anonymous users:

“Netflix was so pleased with the results of its first contest that it announced a second one on Monday. The new contest will present contestants with demographic and behavioral data, including renters’ ages, gender, ZIP codes, genre ratings and previously chosen movies — but not ratings. Contestants will then have to predict which movies those people will like.”

As others have noted this will make it much easier to “de-anonymize” individuals in the collection.

As an experiment, I checked the zip code where I grew up and found that it had about 3900 people in the 2000 census. So, given an age and gender you would have a set of about 40 people. With just a little bit of additional information, one could narrow this to a specific individual.

For example, Narayanan and Shmatikov showed (Robust De-anonymization of Large Sparse Datasets) that this could be done with the dataset from the first Netflix Grand Prize by mining information from IMDB. Think of how much more powerful such attacks would be with the new dataset.

Netflix announced today that BellKor’s Pragmatic Chaos team was awarded the $1M Netflix Grand Prize.

“It is our great honor to announce the $1M Grand Prize winner of the Netflix Prize contest as team BellKor’s Pragmatic Chaos for their verified submission on July 26, 2009 at 18:18:28 UTC, achieving the winning RMSE of 0.8567 on the test subset. This represents a 10.06% improvement over Cinematch’s score on the test subset at the start of the contest. We congratulate the team of Bob Bell, Martin Chabbert, Michael Jahrer, Yehuda Koren, Martin Piotte, Andreas Töscher and Chris Volinsky for their superb work advancing and integrating many significant techniques to achieve this result.”

Netflix announced that it will hold a new Netflix Prize 2 contest with details to be released.

What about the Ensemble’s last-minute entry, the one that seemed to top BellKor’s?

“Team BellKor’s Pragmatic Chaos edged out team The Ensemble with the winning submission coming just 24 minutes before the conclusion of the nearly three-year-long contest. Historically the Leaderboard has only reported team scores on the quiz subset. The Prize is awarded based on teams’ test subset score. Now that the contest is closed we will be updating the Leaderboard to report team scores on both the test and quiz subsets.”

As part of the final submission, teams were required to submit papers describing the approach. Here are the three that the winning team delivered.

• Y. Koren, The BellKor Solution to the Netflix Grand Prize
• A. Töscher, M. Jahrer, R. Bell, The BigChaos Solution to the Netflix Grand Prize
• M. Piotte, M. Chabbert, The Pragmatic Theory solution to the Netflix Grand Prize

The New York Times Bits blog also has an article, Netflix Awards $1 Million Prize and Starts a New Contest.

Today’s Boston Globe has an article on online privacy provocatively titled Project ‘Gaydar’ that leads with a story of an class experiment done by two MIT students on predicting sexual orientation from social network information.

“Using data from the social network Facebook, they made a striking discovery: just by looking at a person’s online friends, they could predict whether the person was gay. They did this with a software program that looked at the gender and sexuality of a person’s friends and, using statistical analysis, made a prediction. The two students had no way of checking all of their predictions, but based on their own knowledge outside the Facebook world, their computer program appeared quite accurate for men, they said.”

I suspect that many will read the article and think that such an analysis can be easily done on their own Facebook information. While I’m not a Facebook expert, I assume that the vast majority of its users employ the default privacy settings which do not allow non-friends to see personal information including gender and the ‘interested in’ attribute, which can be used as a proxy for sexual orientation.

Still, the problem of protecting privacy in online social networking systems is a very real one. The Boston Globe story also mentions work by Murat Kantarcioglu on predicting political affiliations (see Inferring Private Information Using Social Network Data).

“He and a student – who later went to work for Facebook – took 167,000 profiles and 3 million links between people from the Dallas-Fort Worth network. They used three methods to predict a person’s political views. One prediction model used only the details in their profiles. Another used only friendship links. And the third combined the two sets of data. The researchers found that certain traits, such as knowing what groups people belonged to or their favorite music, were quite predictive of political affiliation. But they also found that they did better than a random guess when only using friendship connections. The best results came from combining the two approaches.”

The article also mentions Lise Getoor’s work on discovering private information by integrating work across Facebook, Flickr, Dogster and BibSonomy (see To Join or not to Join: The Illusion of Privacy in Social Networks with Mixed Public and Private User Profiles).

“Those researchers blinded themselves to the profiles of half the people in each network, and launched a variety of “attacks” on the networks, to see what private information they could glean by simply looking at things like groups people belonged to, and their friendship links. On each network, at least one attack worked. Researchers could predict where Flickr users lived; Facebook users’ gender, a dog’s breed, and whether someone was likely to be a spammer on BibSonomy. The authors found that membership in a group gave away a significant amount of information, but also found that predictions using friend links weren’t as strong as they expected. “Using friends in classifying people has to be treated with care,” computer scientists Lise Getoor and Elena Zheleva wrote.”

The Washington Post reports that Flu Trackers Encourage Patients to Blog About It. There was quite a bit of discussion about this back in April with the first wave of H1N1 (swine flu) concerns (e.g., Google flu trends: Web searches as sensors). The article mentions Google Flu Trends and HealthMap, but I was surprised with some of the new ideas people are exploring that the article mentions. Plus, I learned a catchy new term for this: infodemiology.

One idea is to further exploit mobile phone technology.

Boston-based HealthMap’s automated system sends out an hourly Web “crawler” that hunts for flu information in seven languages. Its creators on Tuesday launched a cellphone application called “Outbreaks Near Me” that can alert users to illnesses nearby. “If you move into a zone where there’s an outbreak, your phone would actually alert you,” said John Brownstein, assistant professor of pediatrics at Children’s Hospital in Boston, where HealthMap is based. The application also allows users to send back to HealthMap their own flu alerts.

And another is to recruit a population sample willing to serve as active sensors by reporting their own status and experiences.

Locally, Maryland has launched a “flu watcher” program in which volunteers report their health conditions weekly via the Internet. Project officials say the state is the first in the country to have such a system: the Maryland Resident Influenza Tracking Survey.

“We get people to sign up online and give us their e-mail address,” said Rene Najera, an epidemiologist with the Maryland Department of Health and Mental Hygiene. “They give us their county of residence, their month and year of birth. We don’t get too personal with them. We just want some basic demographics. Every week . . . we send them a survey . . . ‘Did you have any fever? Did you have any cough? Did you have any sore throat in the week previous?’ ” he said. If the answer is yes, more detailed questions are asked. So far, 740 people across the state have signed up.

And the Maryland system is not the only one — see the Australian Flutracking system for another, which gets responses from about 6,000 people.

Researchers at the National University of Singapore have developed a system called FluLog that will use Bluetooth to locate people who had been in proximity to someone who has become infected.

It’s a high-tech version of a process called “contact tracing,” said Mehul Motani of the National University of Singapore’s Faculty of Engineering. Typically, he said “when you have a suspected case, you interview the suspected case, and you ask them: ‘Where have you been? . . . Who have you been in sustained contact with?’ ” The idea is to locate others who might get sick.

Many of these systems have serious privacy issue, of course. But the examples discussed in this article (only some of which are mentioned here) are all voluntary.

It would be great if some of these systems could expose data as RDF making it available as part of the web of linked data.

RAEng report on Social, legal and ethical issues of autonomous systems

The Royal Academy of Engineering has released a report on the social, legal and ethical issues involving autonomous systems — systems that are adaptive, learn and can make decisions without the intervention or supervision of a human.

The report, Autonomous Systems: Social, Legal and Ethical Issues (pdf), was based on a roundtable discussion “from a wide range of experts, looking at the areas where autonomous systems are most likely to emerge first, and discussing the broad ethical issues surrounding their uptake.”

While autonomous systems have broad applicability, the report focuses on two areas: transportation (e.g. autonomous road vehicles) and personal care (e.g., smart homes).

“Autonomous systems, such as fully robotic vehicles that are “driverless” or artificial companions that can provide practical and emotional support to isolated people, have a level of self-determination and decision making ability with the capacity to learn from past performance. Autonomous systems do not experience emotional reactions and can therefore perform better than humans in tasks that are dull, risky or stressful. However they bring with them a new set of ethical problems. What if unpredicted behaviour causes harm? If an unmanned vehicle is involved in an accident, who is responsible – the driver or the systems engineer? Autonomous vehicles could provide benefits for road transport with reduced congestion and safety improvements but there is a lack of a suitable legal framework to address issues such as insurance and driver responsibility.
…
The technologies for smart homes and patient monitoring are already in existence and provide many benefits to older people, such as allowing them to remain in their own home when recovering from an illness, but they could also lead to isolation from family and friends. Some users may be unfamiliar with the technologies and be unable to give consent to their use.”

The RAEng report recommends “engaging early in public consultation” and working to establish “appropriate regulation and governance so that controls are put in place to guide the development of these systems”.

rdf:SeeAlso Autonomous tech ‘requires debate’; Scientists ponder rules and ethics of robo helpers; Robot cats could care for older Britons.

(via Mike Wooldridge)