Find public datasets with the right license, columns and coverage
You need data for a thesis, a dashboard or a model, and search results bring up dead links and datasets with no license. Your todo.is agent searches government open data portals, research repositories and dataset hubs, checks each one's license, size, update date and columns, and can download a sample so you see exactly what you'd get.
The prompt
- Find public datasets for my project: [PROJECT]. I need data about [WHAT DATA] covering [PLACE AND TIME PERIOD]. Search government open data portals, statistics offices, research repositories and dataset hubs. Find 8 to 10 candidates. For each give the name, publisher, link, license and whether it allows [MY USE], format, size, how often it's updated and the last update date, the main columns, and known gaps or quality issues. Download a small sample of the best 3 and show me the first rows. Recommend which one to use and how to combine them if needed. Excel file with the list.
What to change
- [PROJECT]: e.g. "a master's thesis on bike lanes and accidents" or "a dashboard for our city council".
- [WHAT DATA]: e.g. "road accidents with location and severity" or "monthly rents by neighborhood".
- [PLACE AND TIME PERIOD]: e.g. "Spain, 2015 to now" or "US counties, latest year".
- [MY USE]: e.g. "academic research", "commercial use" or "training a machine learning model".
Example result
- Public datasets: road accidents and bike lanes, Spain, 2015 to now
- Use: master's thesis (academic). 9 datasets found, 3 sampled.
- Recommended
- 1. National road accident records · national traffic authority (example)
- • License: open data license allowing reuse with attribution
- • Format: CSV per year · about 100k rows a year
- • Updated yearly, latest full year available
- • Columns: date, time, province, municipality, road type, vehicles involved, bicycle involved (yes/no), severity
- • Gap: exact coordinates only for some years
- 2. City open data: accidents with coordinates · city portal (example)
- • License: open, attribution required
- • Format: CSV and GeoJSON, updated monthly
- • Columns: date, street, latitude, longitude, vehicle types, injuries
- • Gap: one city only
- 3. Bike lane network · city mapping service (example)
- • Format: GeoJSON, updated a few times a year
- • Columns: segment id, type (protected, painted), opening year, length
- Sample (dataset 2, first rows)
- • 2024-03-02 · 08:15 · Calle de Alcalá · 40.42, -3.68 · bicycle + car · 1 minor injury
- • 2024-03-02 · 18:40 · Paseo del Prado · 40.41, -3.69 · bicycle · no injury
- How to combine them
- • Use dataset 3's opening year to compare accidents near a lane before and after it opened.
- • Join datasets 2 and 3 by location (a buffer of about 20 m around each lane segment).
- • Use dataset 1 for the national picture and as a check on city totals.
- Also found
- • 6 more in the Excel file, including a European statistics table (country level only) and 2 datasets with unclear licenses that I don't recommend.
- Before you rely on it
- • Read each dataset's documentation for definitions, like what counts as "serious" injury.
- • Cite the publisher and the download date in your work.
How to do it with todo.is
- Copy the prompt and describe your project, the data you need, the place, time period and how you'll use it.
- Send it in todo.is or by Telegram.
- Your agent searches portals and repositories, checks licenses and freshness, and samples the best datasets.
- Ask it to download and clean the dataset you choose, or to build a first chart from it.
Tips for a better result
- Say how you'll use the data. Commercial use and model training can rule out some licenses.
- Check the last update date. Many datasets on portals were published once and never updated.
- Look for a data dictionary. Column names alone can mislead.
- Ask your agent to save the source link and download date for your citations.
find public datasets: FAQ
- Where can I find free public datasets? Good places are national and city open data portals, statistics offices, international organizations, university research repositories and dataset hubs for machine learning.
- Can I use public datasets for commercial purposes? It depends on the license. Many open licenses allow it with attribution, but some are non-commercial only. Read the license on the dataset page.
- Can the agent clean and analyze the data too? Yes. It can download the data, clean it in Python, join datasets and deliver CSV or Excel files and charts.
- What if no dataset has exactly what I need? Your agent will say so and suggest the closest options, a proxy measure, or the agency to contact for the data.
JavaScript is required to use the todo.is app.