OpenHouse 2017 Goals

Although we're still in November 2016 as I write this, the Data Skeptic team is busy at work formulating plans for 2017. In particular, we're scoping out what our priorities should be for the OpenHouse project next year.

Below, in no order, are the objectives we're thinking are both practical and critical for the project. If you have a perspective on this or want to help, we'd love to hear from you.

Technology components

• Crawler dashboard - Expose a dashboard that describes, in real time, the work done by our crawler.
• Parsing interface - Create an interface where a volunteer can see the rendered and raw HTML for a page which our crawler has cached. Ths interface should also provide a PythonAnywhere window that the contributor can use to write code to extract data from the raw version. The inferface should have functionality to test/check their efforts. Lastly, once the parsing code is written, the versioning and deployment should be managed via continuous integration.
• Scalability - Explore how our API (the only component not currently elastic) can be extended to scale up for high traffic.
• Aggregate API - Extend our API to provide endpoints that return aggregated results and time series data, in addition to our granular responses currently available.

Data collection

• Get active in at least three non-US based countries
• Get at least 100 million unique listings in the United State (at least most recent transaction)
• Capture 50% of the transactions that occur in 2017
• Make contact with the right data people in the top 20 counties in the US and determine how we can best work with them

Press

• Introduce an "Analysis of the month", which will be featured on Data Skeptic via our podcast, blog, and newsletter.
• Boost traffic to gallery.openhouseproject.co
• Get more consumers of our API, ideally hundreds of API queries per day
• Get data journalists to leverage our API in their analysis

Create low hanging fruit for contributors

• Promote easy tasks
• Host working sessions where attendees can get training on technology or methods and start making small contributions
• Devise some reward system for both technical and non-technical contributions