Google Summer of Code Wrap-Up

The Google Summer of Code (GSoC) 2016 is drawing to a close so we thought it best to summarise our experience with GSoC from the mentors’ perspective. We originally came to GSoC through the Mozilla Science Lab. We’ve been having regular calls with Abby Cabunoc from Mozilla, who’s been kind to support openSNP‘s efforts in trying to make open science more accessible. Together we quickly identified a need for more volunteers and people working on the project. From there, we got to the Open Bioinformatics Foundation (OBF), which has been participating in GSoC for years. The OBF kindly agreed to take openSNP under its umbrella, all we needed to do is come up with a few projects, which didn’t take long with the help of Abby (thanks again!!).

We came up with 3 projects that we wanted to tackle for quite a while: 

1. Overhauling our frontend, which has been stuck in the non-responsive dark ages of Twitter Bootstrap basically since the start of openSNP in 2011. Something that’s been rather annoying to many of us over the years.
2. Connecting the Phenotype and SNP information. We do have annotations for genetic variants that basically already tie these genetic variants to the phenotypic data we’re storing. But so far there’s no easy way to go from one to the other. Which limits the use of openSNP for many use cases.
3. Getting more Quantified Self data from Fitness-Tracking devices into openSNP. We’ve been offering support for Fitbit for a while, but nothing for other devices. We wanted to change this and add some more support in general.

For project #1, the frontend work, we could win Mateus, who is largely teaching himself how to work with CSS, JS and all that weird stuff. He’s from Brazil but recently moved to Toronto. See his final blog post on what he achieved.

For project #2 (phenotype <-> SNP link) we could win Vivek who is an Biotechnology student at IIT Kharagpur (India). See his final blog post here. 

For project #3 (more Quantified Self data) we could win Graham, who is an Computer-Science student at the University of Illinois. You can see his final blog post here.

We handled the communication by having a bi-weekly call with whoever had time to show up, which isn’t easy when you have one mentor in Germany, one in Australia, one student in the US, one in Canada, and one in India. It was rare that all students could make time for the call, but the majority of communication was still handled via openSNP’s gitter channel, or via email. Surprisingly, the gitter channel is also where students started to help each other if mentors were still asleep in their respective time-zones. Towards the end of GSoC we increased the frequency of group calls to make sure that all students had support with their deliverables (blog posts similar to this one, code excerpts, etc.). It’s a bit daunting for a first-time GSoC project to accept three students, but we’ve been lucky with three students who were all very well to work on their own, and who always stayed on our radars and checked in with us to share their progress. We’ve also been lucky to have Abby help us with making sure that the projects are well-defined.

Most of the students’ projects are not 100% finished but that’s not the end goal here, what we want is more people to join us on this journey of open science, not to give us finished products and then disappear. We’re confident that the students will keep on collaborating, as we believe that participating in such a small, easy-to-change open source project is a great source of programming experience for novel programmers. If all goes well, openSNP will probably participate as mentors again next year. We’re still unsure how many students we’ll take next year, we were probably very lucky this year. It will depend on how many actual problems we’ll have then. For starters, the gender ratio of people involved in openSNP is still predominantly male, and which open source project is ever truly finished?

If you’re now interested in joining openSNP you can find all of us hanging out in our Gitter channel at https://gitter.im/openSNP/snpr, where you’re most likely to find one of us being awake and ready for a chat. We also want to hold regular office hours via Google Hangouts once per month in the future, stay tuned for that.

Thanks to the OBF for making us part of this great experience and also to our three awesome students! Again, you can take a look at the final blog posts from the participating students here in order of the projects we proposed:

1) Mateus: Front-End Work 

2) Vivek: SNP <-> Phenotype Links

3) Graham: QuantifiedSelf-Support

Our 2015-End-Of-The-Year Newsletter

It’s been a while since we sent out a regular newsletter with the latest updates on what’s happening behind the scenes. Which, it turns out, has been quite a bit in 2015:

Financial support
You can now support openSNP on Patreon and pledge a small monthly amount to help us out in running the service.

Even if you just chip in a single dollar per month, this would already help us a lot. Some of you already do, thanks for that! To give this move some context: Since 2011 we – Helge, Philipp and Bastian – have been running openSNP largely from the money we earn in our dayjobs. We do this because we believe in running a community project around personal genomics.

But with the growth of openSNP we had to move the project to a higher number of larger servers, which means that we have to spend more money as well. This is why we’re now also going for the last step, marrying open source and open science with crowdfunding. Nearly 20 patrons are already contributing, including one of our larger contributors, Seven Bridges.

Hardware updates
So far everything needed to run openSNP was living on a single machine with slow spinning disks — this became more and more unsatisfactory the larger the database grew. We compartmentalized the architecture and moved the different parts to their own virtual servers.

The fact that the new servers are backed by SSD storage brought the most significant improvement in terms of performance, specifically for the database. The background jobs got their own machine as well, so they don’t interfere with other applications by eating up all the resources.

In order to be more flexible in the future, the Rails app is now running within Docker containers, so they can be moved around without as much effort in the future, if needed. If you are interested in joining us in running openSNP: Feel free to get in touch through GitHub or via info@opensnp.org

Phenotype Request: SAT scores
The genetic basis for intelligence has been a hot topic for ages by now. For a primer on the topic you can watch this video. While visiting China and Hong Kong, we met up with Laurent Tellier, who is working at the BGI in Shenzhen. His research is into the genetic basis of intelligence and uses SAT scores as a quantitative proxy. So if you’ve taken the SATs and are willing to share, please help him by entering your variation for those phenotypes:

Phenotype 1
Phenotype 2
Phenotype 3
Phenotype 4

With this we wish all of you happy holidays and some fun science in 2016!

Cheers,
Philipp, Bastian, Lore and Helge for the openSNP team

Support openSNP on Patreon

tl;dr: As openSNP has grown so much we will now really have to upgrade our servers – they’re straining right now already. If you want to chip in, you can support us with a small monthly contribution on Patreon!

Since our last post in April we’ve grown quite a bit, right now we’re hosting ~2100 genetic data sets. Those are largely 23andMe/Ancestry/FamilyTreeDNA genotypings with a few exome data sets. In total this translates to around 1.5 billion genetic variants stored! Add to this all the phenotypes stored in openSNP and we are talking a substantial amount of data. While we are very happy with the growth of openSNP, but at the same time this means that our small server, which is running all of the project, is hitting its limits right now. You will have noticed that using the website can be tedious at times, to the point of your connection even timing out. This is largely because the web server is running on the same machine as the backend, and processing new uploads and maintaining the database backend consumes lots of resources.

To get around those limitations we will have to upgrade our technical infrastructure. Unfortunately doing so will not come for free. So far we have been running openSNP mainly on our own money that we make on our day jobs. This means we are somewhat limited in how much we can shell out to in order to keep openSNP running. As the huge amount of data in openSNP already shows the power of crowdsourcing we are now reaching out to you!

Using Patreon you can support us with as little as $1 per month. We estimate that with around $100 to $150 per month we could substantially improve the speed of openSNP, switching to servers with more RAM, more disk space and faster CPUs. Thanks a lot to those who have become our patrons already. And as usual: Feel free to get in touch any time you have further questions regarding the project at info@opensnp.org!

We would like to again thank all volunteers who’ve made this possible. You rock!

1000 genotypings, and 3 years of openSNP!

In 2011, someone asked me how many genotypes I personally would expect users to upload, and if I remember correctly, I said 30. That was quite the understatement: Just a few days ago on the 30th of May, openSNP received its 1000th genotyping!

On this happy occasion we thank the users and participants for their trust in the project and their continued support and interest.

Since 2011, people have used openSNP in research, art, and their own projects, have written additional software to interact with openSNP’s API, written great comments, and much more. We have published a paper on openSNP a few months ago, which is for most of us the first publication in our careers.

openSNP has come a long way, here are the first three commits from June 2011, with Basti’s oldest on the bottom:

That was nearly exactly 3 years ago! Since then we’ve (among other things) learned to write proper commit messages.

So what does the future hold for openSNP?

1. A better server: We recently received a grant from Bayer HealthCare so we can move to bigger servers, so that the site should load and react much faster. Maybe we can even start hosting bigger datasets?
2. Pre-given phenotypings: One of the biggest problems with openSNP’s data is the high amount of variation in phenotypes entered by users: researchers who want to work with the data still have some manual cleaning to do. We’ve prepared a set of phenotypes for which users can only choose their variation; this should greatly improve the speed with which researchers can start working on the data.
3. Faster parsing: we’ve replaced the Ruby-based genotyping parser by a 99% complete implementation written in Go. So far, it’s much much faster, but still only marginally tested.
4. A variety of smaller things: A stats-page, bug-fixes, genosets, etc. – have a look at the Issues page here.

We thank you for your continued support and interest, and here’s to many more years! If you know of any other project that uses openSNP data, feel free to post it in the comments!

Crowdfunding DTC Microbiomics & Proteomics

We hope that you all had nice holidays and made it safely into the new year. You may already be tired of all the donation campaigns which are frequent in this season and don’t worry, we won’t ask you for money for openSNP. We do however have some ideas on what you could spent your Christmas bonus: Crowdfunding startups and products became en vogue in 2012 and the whole Quantified Self and Personal Genomics movement fortunately has jumped on the bandwagon as well. Right now there are some projects looking for funding on Indiegogo. For example uBiome, which wants to provide insights into the bacteria living inside you, and Talking20, which wants to do for your proteins what personal genotyping companies did for your genome.

We feel that those projects are picking up where the genotyping and genome/exome sequencing efforts of other companies are currently ending and thus might be of interest for you as well (I mean: Who wouldn’t like to get his microbiome sequenced and have regular metabolite tests? Right?). And of course, we hope that we can include sharing capabilities for willing participants of those projects in future versions of openSNP. We have reached out both projects to get some more information about their ideas and future plans of possible collaboriations. Before we let the projects speak for themselves just a standard disclaimer: We’re not involved with any of those projects and are not making any money out of this. We just like the idea of people having more biological data on themselves.

uBiome

Hello, openSNP Community! I’m a PhD student at Oxford, and together with my co-founders from the University of California, San Francisco, I am honored to work with uBiome, the world’s first citizen science effort to map the human microbiome.

The project is funded through crowdfunding at www.indiegogo.com/ubiome. Each gut kit is $69; gut and mouth kits are $139 for both (+$12 shipping outside the United States for each). We plan to send out the kits in May 2012 and return the results on our website once we get the kits back from everyone. Our kits are available in 196 countries. Data will be freely available to those who sign up, and available to the global research community on an “opt-in” basis. In the long run, we hope to integrate with 23andme and other types of genetic and metagenetic data.

So far, our project has garnered over $55,000 in crowdfunding from over 480 participants in less than a month. Participants from twenty different countries spanning four continents have pledged their support, including the United States, United Kingdom, Australia, New Zealand, Canada, Finland, France, Germany, as well as India, Singapore, and Uruguay. We’ve been featured so far in Wired, Venture Beat, the Los Angeles Times, Scientific American, BoingBoing, and syndicated in 160 newspapers around the world through the Associate Press.

What is the microbiome, you ask?

The microbiome are the bacteria that live on and within us. It sounds kind of funny, but all of us are actually covered in helpful germs (or co-evolved symbionts if you prefer).

Like the rainforest, the healthy human microbiome is a balanced ecosystem. The correct balance of microbes serves to keep potential pathogens in check and regulate the immune system. Microbes also perform essential functions such as digesting food and synthesizing vitamins. Some research also suggests that microbial activity influences mammalian mood and behavior. Studies have linked microbiome imbalance to autism, depression, and anxiety, as well as many gut disorders, eczema, and chronic sinusitis. Infant health even appears to benefit from a proper seeding of microbes at birth, with health consequences ranging into adolescence. For some future-thinking commentary on the microbiome, check out this interesting editorial in Science by Leroy Hood.

uBiome brings this cutting edge technology directly to consumers for the first time through citizen science.

We provide participants with a catalogue of their own microbes, detailing the microbial composition of the body and explaining what is known about each genera of microbe. In addition, uBiome compares participants’ microbiomes with scientific studies on the role of the microbiome in health, diet and lifestyle. uBiome also provides personal analysis tools and data viewers so that users can anonymously compare their own data with crowd data as well as with the latest scientific research.

From a small sample on a cotton swab, your uBiome test helps you to learn more about your body, including:

• Diet: Certain gut enterotypes are strongly associated with long-term diets, particularly protein and animal fat (Bacteroides) versus carbohydrates (Prevotella). Maybe you are not sticking to your diet as much as you think.
• Diabetes: Does your gut microflora correlate with people who have diabetes? If you have other symptoms as well, you might want to talk with your doctor.
• Sinusitis: Is your nasal microbiome associated with the profile of chronic sinusitis? Some studies have found that multiple, phylogenetically distinct lactic acid bacteria were depleted concomitant with an increase in the relative abundance of a single species, Corynebacterium tuberculostearicum.
• Alcohol consumption: Do you drink a lot of alcohol? If your gut profile clusters with heavy drinkers, you might want to consider cutting back on the booze.
• Bowel conditions: Do you have Irritable Bowel Disorder (or any other bowel condition)? You may want to purchase our specially designed kit and survey for bowel disorders.

Please join us and help us spread the word about this project. The more people that contribute, the more we can all learn about our health, and contribute to the advance of SCIENCE!

Talking20

Talking20 is a simple blood test that can be ordered online and received in the mail. It’s quick and easy! Results are delivered online in secure profile, and can even be viewed from a smartphone. We are ready to gather data and design experiments that will help us help you learn more about what’s going on inside your body!

Get involved with the Talking20 project by ordering a basic kit and sending it in. Tracking your biological data will help you view changes taking place inside your body, and it will help us design new tests that are useful to everyone. Want to know what eating a burger does to your cholesterol? Interested in tracking stress reduction over the course of a month-long yoga course? We are and we need your help. Your body is talking to you, let’s find out what it’s saying!

This is how Talking20 is measuring your metabolites:

Talking20 is taking proven technologies: (1) dried blood spot collection (done for every newborn in developed countries) and (2) mass spectrometry (used in labs everywhere), and innovatively bringing them together. We believe that actionable information about our bodies are available by looking at our proteins and metabolites. That’s where we need to be looking. Danny Hillis of Applied Minds does a amazing job of explaining this concept in his talk at TEDMED.

Mass spectrometry is a very sensitive method for counting molecules from blood, and is actually used as the ‘gold standard’ to set up hospital tests. With a bit of extra work, mass spec can also be used to measure every other molecule too. Without visiting a lab or even leaving your home, you can now just put a few drops of your blood on a card and mail it to us at room temperature from anywhere in the world! We can then do the analysis and tell you your results wherever you are!

Talking20’s commitment is to make this testing as economical as possible, so you can do it whenever you want. Heather has personally tracked her hormones and cholesterol every few days to learn more about her own life events and diet. Talking20 is also on the verge of completing a cortisol test so we can start watching stress levels in response to different types of exercise!

At Talking20, we want to make many different tests available, so you can start finding out about what is happening inside you. We are offering five of these ‘biomarkers’, to measure your cholesterol, vitamin D, estradiol (estrogen), progesterone, and testosterone. This would normally require you go into the doctor, then into a lab to get a blood draw, and then back to the doctor, and would likely cost $300-500! We believe we can do better!

If you want to support them you should visit their Indiegogo page.

Support for 23andMe-exome-data

In the last weeks some people contacted us because they wanted to upload their exome-data to openSNP. While we started out with a focus on SNP-data (mainly because it’s much more abundant) providing support for exome-data has been on our ToDo-list for a couple of month now, so we did some work on it. Today we start with support for the data which is generated by 23andMe through their Exome-service.

So if you’ve got your exome sequenced through 23andMe you can now upload it to openSNP as well. Unfortunately there are some drawbacks right now:

1. We only support upload for the VCF (Variant Call Format) and not for the BAM-file which contains the raw reads as well.
2. We won’t parse all of your data. We will scan your exome for those SNPs which are already in the openSNP-dataset, but we won’t add any new ones to the web interface and won’t make other variants available for browsing on openSNP in the near future.

There are two main reasons for this limited feature-set:

1. openSNP is our hobby project and we run with limited financial and computational resources. Basically right now we don’t have the computational power and storage to store and parse complete exome-BAM-files to deliver any benefit.
2. Mining literature for known SNPs is easy, because a unique identifier exists for each SNP. This identifier is used in most publications and is seldom used for other purposes than referencing the SNPs. For Copy Number Variations etc. this isn’t that easy and we still haven’t had a good idea to get around this.

Nevertheless: If you’ve got your exome sequenced through 23andMe you can now upload your VCF-file. We will search for all known SNPs and make your genotypes available on the web-frontend of openSNP. And of course the complete VCF-file will be downloadable for anyone who is interested in it. Have fun uploading your data. And as always: Let us know if something doesn’t work as expected or if you have any further ideas for us.

Replies to all genotyping-applications have been sent

It took us some time to read through all your great applications and it was a hard task to decide which applications we should consider for the free genotypings. After heavy discussions and bargaining we had two things: a list of successful applications and a short list of applicants that would get a 23andme-kit if there would be a DNADay-discount with 23andMe this year.

By now all applicants should have received an email from us which either brings the good or bad news. If you haven’t received an email you should take a look into your spam-folder (look for a message from info@opensnp.org). A handful of applicants either had some typo in their address or have closed down the account they used to register for the application. If you haven’t heard from us at all this might be the reason.  In this case, please contact us again and tell us your old or original e-mail-address.

For everybody on the shortlist: Unfortunately 23andMe doesn’t offer discounts this year, so we can’t consider your applications. We’re really sorry about this as we really would like to genotype all of you. Drop us an email if you want to get notified about a potential follow up free-genotyping. We’re still trying to get some funding for this.

Please give us your postal address if your application was successful. We need it to order you a spit kit.

Thanks again to everybody who participated.

Cheers,
the openSNP-team

Update on the free genotypings and new features on openSNP

The free genotypings

This night the deadline for free genotypings passed, and we are overwhelmed by the amount of responses. In less than 24 hours after the first blog post over 200 people sent us their applications, and in total over 450 people applied. For us the real work now begins, and we will do our best to go through all the applications, select the most interesting, and then contact every applicant. 

 

Entering Phenotypes

We also used the past weeks to implement some more features into openSNP. This time we focused on end-users entering their data into openSNP, especially on entering phenotypic variations. Entering a lot of phenotypes was a bit cumbersome and took too much time. Now, registered users will find a subtle change to their dashboard: The tab which shows phenotypes you haven’t yet provided now features a small button that allows you to easily fill in your information.

 A modal window (like the one in the screenshot above) will appear and you can choose your variation from the list of already known answers, or you can choose to add a new one. This should make entering data a lot faster. 

 

Phenotype Recommendations

We also added a recommendation engine for phenotypes, which can be found on individual phenotype pages. An additional tab will show you which phenotypes have most frequently also been entered by users who have entered variations on the page you are looking at. 

Additionally, you can use this system to get recommendations about similar phenotypes and even variations. If you enter your variation for a phenotype through the old phenotype view you will get a similar picture to this: 

 The top row shows you up to three phenotypes and the variation which is often entered by users who have provided similar variations to you. So lets’s say you’ve just entered that your Body Mass Index is above 30, the system might point out that people with this variation frequently have entered that they have a high blood pressure. 

 

The lower row shows you up to three phenotypes which are often entered by users who have entered any information about similar phenotypes as you, irrespective of the variation provided. For example, if you’re interested in visible traits and have just provided information about your eye colour you will see that people who are interested into this also provide information about their hair and skin colour. 

 

Given the rising numbers of different phenotypes which you can enter in openSNP, this should help you to find those that are of interest of you. You might have noticed that you wont get those recommendations if you are using the new quick-entry-feature on your dashboard. Our reasoning for this: Using the dashboard you can enter a large amount of variation in rapid succession. Here, displaying recommendations after each phenotype entered would slow you down. But if you like to “browse” and don’t want to rapidly enter large amounts of data, you might benefit from having some recommendations what might also be of interest for you. 

 

How do you feel about this? Would you like to see an option to get those recommendations also after entering variation using the dashboard? 

Apply now for a free genotyping

At the end of last year we announced that we’ve got some funding from the German WikiMedia foundation to get more people – who are willing to share their results – genotyped. We have now settled on a process that should allow us to perform the project without too many problems. Starting today, you can apply for one of the free genotypings. The deadline for applications is Sunday, 03/25/12 23:59 o’clock, so you still have some time to think about an application. In the two weeks following the deadline, we will select as many participants as we can afford to get genotyped using the 5000 Euros we received from Wikimedia. We’ll get in contact with everybody who has sent an application to let all applicants know whether their application was successful or not.

The genotyping will be done through 23andMe. We will order you a gift kit which will be delivered to your address. These gift kits include prepaid access to the 23andMe website for 12 months, so you can check up on the latest findings about your genetic variation as well. After this 12 month period, those features will expire automatically, you don’t have to cancel any subscriptions.

Our application form contains some standard questions (Where do you live? Does 23andMe deliver to your country? etc.) but also some details about your motivation, why you want to make your dataset available to the public and why your data might be of great interest (For example: Do you have a rare disease where research is lacking?). Additionally, we will also try to get people genotyped who are currently under-represented in publicly available data sets. Most data up to now is from WEIRDs: Western, Educated, Industrialized, Rich and Democratic people (most are probably male, too).

We would like you to deposit the final raw data, which you will then be able to download from 23andMe, into the openSNP database, ideally along with some phenotypic information about yourself. So please think about the possible consequences which may arise by doing so before you apply for one of the genotypings. The application process has some questions about possible consequences as well (just so we get a feeling of whether you know what you are doing). If you get your results, but then find the results too problematic to publish: That is fine. We are aware of this possibility and while it would suck for us as it means less data, you are the one who has the last word in this matter. Some information that might make you a bit more comfortable with the idea of sharing data: We won’t release the names of any applicants (whether successful or unsuccessful) and you can sign up to openSNP using a pseudonym, plus we don’t log any IPs used to access openSNP.

tl;dr

We offer you the chance to get genotyped through 23andMe for free if you are willing to share the data with the public. Here’s the planned schedule:

• Until 03/25/12 23:59 o’clock you can apply for a genotyping using this application form
• We select the lucky winners between 03/26/12 and 04/08/12 and get in contact with every applicant.
• Mid-April: You should receive the 23andMe-kits in your mail.
• End of May: You should receive the results of the genotyping, so you can upload the results to openSNP.

If you’ve got any questions regarding the application process, the schedule etc., just let us know using the comments or write us an email to info@opensnp.org. We will try to answer all of your questions as fast as possible.

Good luck,
wishes your openSNP-team!

Videos on openSNP & DAS-support

Another thing we have been procrastinating for far too long: Creating videos on the idea of openSNP and some screencasts that show how you can use openSNP to enter data about yourself and how you can get the data out again for your own research. So here we go. The first video is a small “self-interview” I did to tell you a bit how we started, what you should keep in mind privacy-wise before starting to use openSNP etc.

The next video shows you how you can use the openSNP-frontend to enter your phenotype-data, what kind of information the individual SNP-pages can show you and how you can subscribe to the openSNP-RSS-feeds to be notified about the latest genotyping-files etc. A small new feature which is missing from the video as we implemented it after recording the video: The news-page features a tab that includes the latest publications on the SNPs we have in the database. And for all info-junkies: You can also subscribe to the latest publications using RSS.

The last video shows how you can query the APIs which we have implemented. Bonus-Content: This video includes the first preview on how you can use the Distributed Annotation System to visualize individual genotyping files! In short: You can use http://opensnp.org/das/sources to get a list of all DAS-sources we have with openSNP. Each source represents all SNPs we have of a single user, regardless of how many genotyping files a user has provided.

If you want to use a DAS-source in a genome browser, for example in MyKaryoView you can use the features-commmand of DAS. The link for this is http://opensnp.org/das/$user_id/features, where you have to replace $user_id by the ID of the user you are interested in. If you want to query SNPs between chromosomal positions using DAS you can use http://opensnp.org/das/$user_id/features?segment=$chromosome_name:start,stop. So http://opensnp.org/das/1/features?segment=1:1,1000000 will give you all my SNPs on Chromosome 1 between position 1 and 1000000.

If you want to see an example for a visualization using DAS look at the video below. The DAS features are still experimental. I will attend this DAS workshop to get some help with the final implementation, so if you have suggestions: Please let us know!

Enjoy playing around with this features. As usual: Let us know if you find any bugs we have missed!