Understanding ClinicalTrials.gov

ClinicalTrials.gov contains one of the most important public biomedical datasets in the world. And yet… using it well still requires:

  • Knowing the API
  • Understanding filter syntax
  • Manually constructing advanced queries
  • Interpreting dense protocol output

I wanted to change that. So I built RareTrials MCP Explorer — a system that lets you query ClinicalTrials.gov using plain text and easy interfaces.

My Leap into Data Science

I’ll never forget the feeling of giving my two weeks notice at my cushy and comfortable Data Scientist job at Biogen. I remember walking out of the confines of the corporate building into Kendall Square with a sense of freedom, excitement, and absolute terror. The checks would stop coming in just a few weeks.

AI Development Standards

On October 30th, the White House issued an Executive Order outlining various new standards to ensure the safe, secure, and trustworthy development and use of Artificial Intelligence. The next day, DJ Patil, former U.S. Chief Data Scientist, posted the below on LinkedIn: "Since we already have a tool built to do exactly this, CHALLENGE ACCEPTED!"

The Challenge of Information Overload

In today’s world, staying updated with the latest news and information is crucial. One of the best sources for real-time news is the internet, where we can access a wide range of articles, blogs, and other content. However, manually sifting through all this information can be a daunting task. This is where the power of data aggregation tools comes in.

Analyzing Super Bowl 54

Super Bowl 54 is only 2 days away so we decided to analyze the upcoming game by pulling some data from ESPN.com. We used R to scrape the data from the San Francisco 49ers and Kansas City Chiefs stats pages. The data were then cleaned and transformed slightly to make ingestion into Tableau Public simple.

NFL Season Insights

The NFL season is back in full swing. If you’re a sports and data nerd like myself, you’re ready to flex some of your analytical skills to try and beat all your friends in your fantasy leagues. Or you can take a crack at Daily Fantasy, which involves picking players and forming a team based on statistical performance.

Game of Thrones Viewer Reactions

Game of Thrones has finally wrapped up after 8 seasons. Naturally, watchers have been outspoken about their opinions of the series finale and the final season altogether. Now that we’ve had a few days to recover, we wanted to analyze watcher reactions and see if we could add some structure to our understanding of these sentiments.

The Rising Demand for Data Skills

Demand for Data Skills Continues to Rise. Over the past 10 years, we’ve seen an explosion in the generation of data, coupled with the rise of the data scientist to help make sense of this ever-expanding corpus of information. Technological advancements in AI, machine learning, and computational power have increased the impact that these scientists have.

Issues with Manual Data Entry

Manual data entry still represents a significant portion of data collection processes across a variety of industries. Any company that assigns humans to conduct data collection will see data entry errors. These errors may appear insignificant, such as a code entered as “ABCD” instead of “AVCD”, but they can be very costly. Remote data entry solutions can help mitigate these issues.

Innovative Data Science in Pharma

During the Data Science Plenary Panel at Bio-IT World 2018, Tanya Cashorali of TCB Analytics highlighted some innovative use cases for data science in the pharmaceutical and biotech industries. These use cases span drug manufacturing, clinical enrollment, and bioinformatics. For an example of one use case, here’s an in-depth look at how we helped Biogen.