Personal Data Frontier – Launching in January

Posted by Kaliya on Monday, December 26th, 2011

PDEC is please to announce the launch of its premier publication The .

It is the essential monthly guide written by and for personal data ecosystem pioneers. Our subscribers are the ground-breaking technologists, product development enterprises, data entrepreneurs, and policymakers who are shaping and leading this rapidly-evolving sector. In each monthly issue, our experienced reporting team and respected guest contributors track, distill, and make sense of the 100+ emerging standards, rapidly growing number of listservs and companies, analyst reports, social media buzz, news, and insider talk.

Stay Apprised of this Emerging Territory
A small group of technology visionaries has toiled in the shadows for years to illuminate and shape an important, emergent market niche: The personal data ecosystem. As disastrous privacy breaches become the norm, there’s growing pressure from many quarters to shift the locus of control over personal data from institutions to individuals. This profound change — like the revolutions initiated by automobiles, television, personal computers, and the web — promises to transform every aspect of our day-to-day lives, including how we buy and vote, how our children navigate in cyberspace, and how business and governments across the globe engage with people and their data.

Get Access to the Most Credible Resource Available Without Overload
Published by PDEC, the Personal Data Ecosystem Consortium, Personal Data Frontier is the comprehensive source for information in the emerging personal data services space, offering in-depth analysis and news no other publication does, or can. The Frontier provides invaluable leverage to those in the vanguard of developing and delivering solutions and services in the personal data ecosystem. The Internet Society (ISOC) is funding PDEC’s research on the industry practices and trends the Frontier will regularly examine and interpret. Each monthly issue boosts your sector intelligence with insightful, broad-based, and knowledgeable reporting. We do the legwork so you can focus on making the most profitable use of what we reveal in your sphere.

Subscribe Now and Keep Your Edge 

Posted by David Strom on Monday, December 26th, 2011

I spent part of this week with Hilary Mason, one of the smartest people that I know in Big Data. She works as the Chief Scientist for Bit.ly and has a wealth of skills at her fingertips that bridge computer science and mathematics. Plus, she is used to facing largely male audiences and just being the smartest person in the room. She was speaking at this week, which should definitely be on your radar for next year if you are interested in this topic or want to broaden your programming skills.

Sponsor

Editor's note: This story is part of a series we call Redux, where we're re-publishing some of our best posts of 2011. As we look back at the year - and ahead to what next year holds - we think these are the stories that deserve a second glance. It's not just a best-of list, it's also a collection of posts that examine the fundamental issues that continue to shape the Web. We hope you enjoy reading them again and we look forward to bringing you more Web products and trends analysis in 2012. Happy holidays from Team ReadWriteWeb! Mason outlined in a series of workshops the tools you need to get started with manipulating Big Data and understanding the basics of machine learning, something she does everyday as she sifts through each one of those shortened URLs that we all create furiously. (You can ) You know when she says, "this is a hard problem" that she is really saying "this is a problem that I haven't yet figured out the best answer to." To each problem, her credo is Obtain, Scrub, Explore, Model, and Interpret. I'll review each of these steps. The first step is , and for Mason it is a Linux machine with a variety of tools on it that you can find on her Github page linked above. She is a Python programmer, and so this reflects that interest. She uses Python with JSONview's Chrome extension, NLTK, numpy, Pycluster, hcluster, and mathplotlib. You can use most of these tools on other OSs too. Second, you need to obtain a few test data sets that you can start to manipulate. Even if you aren't drinking out of the Bit.ly data fire hose, there are ways to get access to lots of great data around the Internet. Mason mentioned a few places, including:

• The New York Times. Each and every article that is posted on the main NYT Web site going back dozens of years has oodles of metadata and tags galore. Just view the source on any news article to see how the human editors have classified it. You'll need to start off by and then select "Article Search API"
• Mason has put together several
• that he has collected, and there is always the epic reference

Some of these datasets have gotten almost urban legend status, such as the email collection that was part of all the Enron court cases: this is useful for testing anti-spam programs, even though it is several years old. Because of how people use Bit.ly, Mason said that they usually see malware and other bad stuff several hours before anyone else has picked it up around the Internet. Third, you need to start thinking about how to make your data sets smaller. "Big Data usually refers to a data set that is too big to fit into your available memory, or too big to store on your own hard drive, or too big to fit into an Excel spreadsheet," says Mason. This is the "scrub" section. The smaller the dataset, the easier it is to manipulate and analyze. Now comes the fun part, exploring your data. You want to ask questions and figure out patterns. If you are using the NYTimes data for example, you could look at words that are most frequently used to describe political candidates, or are particular words more often used in the technology section than in the sports section. Next comes the mathematical modeling portion of the program. If you don't have a lot of depth in probability or statistics, you are going to need some help here. Mason rolls off the math almost as if she is speaking fluently in a foreign language. And given that I was an undergraduate math major, I can only understand your own frustrations here when you see something with the Greek Sigma sign (hint: that means a sum of things). She let slip the words "fourier transform" which brought me to the Wikipedia definition before I could try to remember what it was. But the essence here is to write code to get the answers of the questions that you are asking in your explorations. Finally is interpretation. You want to put the answers that you obtained from your modeling into the context of why they are important. You may need to do some visualizations or reporting so that your results can be understood. Or you may choose to omit this part if your answers are sufficient for your particular purposes. Mason mentioned her spam-fighting tactics that she uses to cleanse the shortened URLs from evil doers. "As long as our routines seemed to be working, that was good enough for us and so we just stopped," she said. Part of this process is understanding when you have the best answer that you are going to have to your questions, "knowing when you have won" as she says. If you are looking to learn more, a good place is to sign up for one of The classes start October 10 and continue until mid-December. They are free and you submit homework assignments and do everything that you would normally do in a typical CS class, including getting help from teaching assistants too. Mason is lucky to have such a large playground to operate in, and we are lucky when we can sit at her feet and try to understand the totality of her experience. I hope I have given you a taste for the world of Big Data here and some of the best ways to get started at your own analysis. And if you need more motivation, she tells me that just about everyone is hiring their own "data person" these days, so if you get good enough at you probably have your pick of employers for years to come.

Posted by Gerrit Visser on Saturday, December 24th, 2011

Jon Leibowitz (FTC) : “Last December, FTC staff issued a draft privacy report reimagining how we can protect consumers in an increasingly digital and mobile age; we expect to put out a final version of that report in the next month or so. The preliminary report recommended that companies build privacy into their systems at the design phase – what we call “privacy by design”; simplify the ability of consumers to exercise choices; and improve transparency of information practices. As one example of simplifying choice, the report recommended implementation of Do Not Track, a system we envision empowering consumers (and not the government) to take control of the data companies collect about us online”. read the full text also see the of the Face Facts Forum (Dec. 8, 2011)

Posted by Gerrit Visser on Saturday, December 24th, 2011

Speaking at a Dec. 15 Capitol Hill forum on children’s and teens’ online privacy, Federal Trade Commission Chairman said that the agency is recommending that the Children’s Online Privacy Protection Act (COPPA) expand the definition of personally identifiable information. Leibowitz explained that he supports expanding the definition of “personally identifiable information” to include geolocation information, photos, videos, IP addresses, and similar items found on computers or mobile devices. by see for more information

Posted by Dissent on Thursday, December 22nd, 2011

Rachel Grunberger writes: The Senate Judiciary Subcommittee on Privacy, Technology, and Law recently held a hearing to discuss federal enforcement of the Health Insurance Portability and Accountability Act (HIPAA) and the Health Information Technology for Economic and Clinical Health (HITECH) Act, entitled “Your Health and Your Privacy: Protecting Health Information in a Digital World.” In [...]

Posted by Dissent on Wednesday, December 21st, 2011

Adam Ghassemi reports: Last week, while unloading debris at the Sumner County Resource Authority a woman found a lot more than just trash. “It’s totally reckless,” said a Sumner County woman who doesn’t want her identity revealed. She says she hopes what she did protects the identities of countless others. “It’s just unbelievable what could [...]

Board Changes at PDEC

Posted by Kaliya on Tuesday, December 20th, 2011

Mary Hodder announced her resignation from the board to shift her attention to . We are grateful for her service to the organization and wish her all the best in her future endeavors.

Posted by Drummond Reed on Tuesday, December 20th, 2011

Mike Schwartz, CEO of Gluu and one of the hardest working members of the OASIS XDI Technical Committee, has started on the Gluu blog. It lends gentle and beautiful insight into this new semantic data format and protocol.

Resigning my Post as Chairman at PDEC

Posted by Mary on Monday, December 19th, 2011

Effective immediately I’m resigning from the Personal Data Ecosystem Consortium’s board after nearly 14 months as Chairman.

As many of you might have seen, we accomplished a lot in the last twelve months, written up in our Part I, II and III end of year summary newsletters (at our blog here, here and here).

In this first year of PDEC, we published papers, spoke at events and contributed to various endeavors in the personal data discussions happening on the web and in person around the world.

I’m personally very committed to a world where individuals drive their own data and I’m very proud of the work we did at PDEC, which is focused on companies and how those companies can build for a personal data ecosystem.

In the past couple of months, I’ve also worked to create a new org: , with about eight other folks, where the org is for Individuals only, no companies may join. Customer Commons looks at markets and data from a strictly individual point of view. I believe that it’s a conflict of interest to work on both organizations (which represent either individuals or companies). Therefore, I want to see Customer Commons get up and running, and I realize I can’t remain at PDEC, which represents the company perspective on the personal data ecosystem.

So for the foreseeable future, I’ll be working on the same personal data issues, but from this Individual perspective, at Customer Commons. I wish Personal Data Ecosystem Consortium all the best.

~ Mary Hodder

Posted by My Digital Footprint on Friday, December 16th, 2011

Vice-President of the European Commission responsible for the Digital Agenda Data is the new gold Opening Remarks, Press Conference on Open Data Strategy Brussels, 12th December 2011 SPEECH/11/872 Brussels, 12th December 2011 Good morning everybody it is a pleasure to be here for such a positive announcement. Just as oil was likened to black gold, data takes on a new importance and value in the digital age. Web entrepreneurs assemble and sell content and applications and advertising, based on data. With those efforts they make our lives more convenient and they keep authorities accountable. They live on data, and increasingly so do the rest of us. (NO: WE LIVE ON THE ANALYSIS OF DATA) Today, I am proud to present an Open Data package that can drastically increase the possibilities for those web entrepreneurs; the opportunities of businesses, journalists, academics and all citizens, in fact, to generate new and rich content. (YIPEE) Today's package will radically shake-up how the EU institutions and most public authorities in Europe share their data. Public data, generated by all administrations in Europe will become automatically re-usable. It will feed new applications and services. It will change the way administrations are working – for the good of the administrations and the people they serve. This is a victory for those of us who believe that the best way to get value from data is to give it away. It is also, by the way, a vote of confidence in the people of Europe. We trust you will do good things with this data – data you already paid for. And it is recognition for the many heroes who have shown us already what is possible through open data. Heroes like the governments of UK, Denmark and France, to people like Tim Berners-Lee and organisations like the Open Knowledge Foundation. Today we are taking their ideas and proposing them as the rule rather than the exception. We are putting the Commission and Europe at the top of the class. We calculate that public sector information already generates €32 billion of economic activity each year. This package would more than double that - – to around €70 billion. That is not coffee money. That is, indeed, a badly needed boost to our economy. The UK, Denmark and France - who last week opened "data.gouv.fr" – are doing excellent work. When the Danish Enterprise and Construction Authority (DECA) lowered fees, the number of users of its information went up by ten thousand percent (10,000%)! This generated four times the tax revenue lost from fees charged for data. Another example is "openspending.org" one of thousands of examples of how citizens can learn about how their taxes and spent, or how big their national debt is. Next, I want to mention WheelMap – I met the founder of this application last week. He won an accessibility award, and rightly so. It is amazing. Wheelmapcombines public data, and user-supplied data, to give disabled people a clear idea of the best and worst places for them. It helps build the case for greater accessibility. All of you too can benefit as journalists, as The Guardian and others have shown through data journalism. Before I go to questions, I want to remind you of the two strands of today's package: First, the Commission is opening up its vault starting in 2012 During the first half of 2012 the Commission's own data will be available free, open and easily usable on computers from a single portal. We will be pushing the other agencies and institutions to join us. We are practicing what we preach. We will dare the Member States to improve on us Second we are proposing to harmonise the way Member States will make their public data re-usable. This should make it possible for business to gain the benefits of the Single Market. This is a revolution. Instead of needing complicated authorisations, you will be automatically allowed to re-use the public data you need. Instead of high charges for this data, fees will be limited to marginal costs. This will boost business opportunities very substantially. We are also proposing to include very valuable cultural material, from libraries, archives and museum. In these cases, re-use will be optional and not free of charge unless the museum chooses to the free option. This exception will give protection the cultural institutions which have particular commercial vulnerability. The revised Directive will need approval from the Parliament and Council. But the real message I want to send to public authorities today is: don't wait for this package to become law. You can give your data away now – and generate revenue and jobs, and even save money from the better information and decisions that will flow. I also say to private businesses: open their data to generate new services. (MISSING THE POINT, GOVERNMENT DATEA WEHAVE PAID FOR, PRIVATE DATA VALUE IS IN THE ANALYSIS – THIS IS IP, AND THAT IS PROTECTED BY OTHER EU LAW) In short, ladies and gentlemen, my message today is that data is gold. We have a huge goldmine in public administration. Let's start mining it. (VALUE IS IN THE REFINEMENT NOT THE DATA!) Start releasing your data now. Join the future. Join the growth. |