{"author":"Jovian.ml","author_email":"hello@jovian.ml","bugtrack_url":null,"classifiers":["License :: OSI Approved :: MIT License","Operating System :: OS Independent","Programming Language :: Python :: 3"],"description":"# opendatasets\n\n`opendatasets` is a Python library for downloading datasets from online sources like [Kaggle](https://www.kaggle.com/datasets) and Google Drive using a simple Python command. \n\n\n### Installation\n\nInstall the library using `pip`:\n\n```\npip install opendatasets --upgrade\n```\n\n### Usage - Downloading a dataset\n\nDatasets can be downloaded within a Jupyter notebook or Python script using the `opendatasets.download` helper function. Here's some sample code for downloading the [US Elections Dataset](https://www.kaggle.com/tunguz/us-elections-dataset):\n\n```\nimport opendatasets as od\ndataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'\nod.download('https://www.kaggle.com/tunguz/us-elections-dataset')\n```\n\n`dataset_url` can also point to a public Google Drive link or a raw file URL.\n\n### Kaggle Credentials\n\n`opendatasets` uses the [Kaggle Official API](https://github.com/Kaggle/kaggle-api) for donwloading dataset from Kaggle. Follow these steps to find your API credentials:\n\n1. Sign in to [https://kaggle.com/](https://kaggle.com), then click on your profile picture on the top right and select \"My Account\" from the menu.\n\n2. Scroll down to the \"API\" section and click \"Create New API Token\". This will download a file `kaggle.json` with the following contents:\n\n```\n{\"username\":\"YOUR_KAGGLE_USERNAME\",\"key\":\"YOUR_KAGGLE_KEY\"}\n```\n\n3. When you run `opendatsets.download`, you will be asked to enter your username & Kaggle API, which you can get from the file downloaded in step 2.\n\nNote that you need to download the `kaggle.json` file only once. You can also place the `kaggle.json` file in the same directory as the Jupyter notebook, and the credentials will be read automatically.\n\n### Some interesting datasets\n\nYou can find interesting datasets on Kaggle: https://www.kaggle.com/datasets\n\n*You can also create a new dataset on Kaggle by uploading a CSV file here: https://www.kaggle.com/datasets?new=true (make sure to keep your dataset public, otherwise it will not be downloadable)*\n\n- Video Games sales: https://www.kaggle.com/gregorut/videogamesales\n- World University Rankings: https://www.kaggle.com/mylesoneill/world-university-rankings\n- Netflix Tv shows and Movies: https://www.kaggle.com/shivamb/netflix-shows/notebooks\n- StackOverflow Developer Survey: https://www.kaggle.com/stackoverflow/stack-overflow-2018-developer-survey\n- Google Play Store Android Apps Data: https://www.kaggle.com/lava18/google-play-store-apps\n- Indian Stock Market Data: https://www.kaggle.com/rohanrao/nifty50-stock-market-data\n- Indian Air Quality: https://www.kaggle.com/rohanrao/air-quality-data-in-india\n- Worldwide Covid-19 Cases: https://www.kaggle.com/imdevskp/corona-virus-report\n- USA Covid-19 Cases: https://www.kaggle.com/sudalairajkumar/covid19-in-usa\n- US Election Results (2012): https://www.kaggle.com/tunguz/us-elections-dataset\n- US Stock Market: https://www.kaggle.com/borismarjanovic/price-volume-data-for-all-us-stocks-etfs/\n- Crop production in India: https://www.kaggle.com/srinivas1/agricuture-crops-production-in-india\n- Agricultural raw material prices: https://www.kaggle.com/kianwee/agricultural-raw-material-prices-19902020\n- Agricultural land values: https://www.kaggle.com/jmullan/agricultural-land-values-19972017\n- Digital payments in India: https://www.kaggle.com/lazycipher/upi-usage-statistics-aug16-to-feb20\n- US Unemployment Rate Data: https://www.kaggle.com/jayrav13/unemployment-by-county-us\n- India Road accident Data: https://community.data.gov.in/statistics-of-road-accidents-in-india/\n- Data Science Jobs Data:\n - https://www.kaggle.com/sl6149/data-scientist-job-market-in-the-us\n - https://www.kaggle.com/jonatancr/data-science-jobs-around-the-world\n - https://www.kaggle.com/rkb0023/glassdoor-data-science-jobs\n- Youtube Trending Videos: https://www.kaggle.com/datasnaek/youtube-new\n- Asteroid Dataset: https://www.kaggle.com/sakhawat18/asteroid-dataset\n- Solar flares Data: https://www.kaggle.com/khsamaha/solar-flares-rhessi\n- F-1 Race Data: https://www.kaggle.com/cjgdev/formula-1-race-data-19502017\n- Automobile Insurance: https://www.kaggle.com/aashishjhamtani/automobile-insurance\n- PUBG video game matches: https://www.kaggle.com/skihikingkevin/pubg-match-deaths\n- CounterStrike GO (video game)\n - https://www.kaggle.com/mateusdmachado/csgo-professional-matches\n - https://www.kaggle.com/skihikingkevin/csgo-matchmaking-damage\n- Dota 2 (video game): https://www.kaggle.com/devinanzelmo/dota-2-matches\n- Cricket One-Day Internationals Data: https://www.kaggle.com/jaykay12/odi-cricket-matches-19712017\n- Cricket Indian Premier League Data: https://www.kaggle.com/nowke9/ipldata\n- Basketball (NCAA): https://www.kaggle.com/ncaa/ncaa-basketball\n- Basketball NBA Players Stats: https://www.kaggle.com/ncaa/ncaa-basketball\n- Football datasets: \n - https://www.kaggle.com/martj42/international-football-results-from-1872-to-2017\n - https://www.kaggle.com/abecklas/fifa-world-cup\n - https://www.kaggle.com/egadharmawan/uefa-champion-league-final-all-season-19552019\n- Hotel Booking Demand: https://www.kaggle.com/jessemostipak/hotel-booking-demand\n- New York Airbnb listings: https://www.kaggle.com/dgomonov/new-york-city-airbnb-open-data\n\n\nOther sources to look for datasets: \n- [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml/index.php)\n- [awesome-public-datasets](https://github.com/awesomedata/awesome-public-datasets)\n- [Google Dataset Search](https://datasetsearch.research.google.com)\n\n*If you use an external source other than Kaggle, you'll create a new dataset on Kaggle by uploading a CSV file here: https://www.kaggle.com/datasets?new=true (make sure to keep your dataset public, otherwise it will not be downloadable using `opendatasets`)*\n\n\n\n## Curated Datasets\n\n`opendatasets` also provides some curated datsets that you can download by passing the Dataset ID to `opendatasets.download`. Here's an example:\n\n```\nimport opendatasets\nopendatasets.download('stackoverflow-developer-survey-2020')\n```\n\nThe following datasets are available for download.\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n

Dataset ID	Description	Source
`stackoverflow-developer-survey-2020`	Stack Overflow Developer Survey 2020	\n Stack Overflow\n
`owid-covid-19-latest`	Covid-19 Stats by Our World in Data	\n Our World in Data\n
`state-of-javascript-2016`	State of Javascript Annual Survey 2016	\n StateOfJS\n
`state-of-javascript-2017`	State of Javascript Annual Survey 2017	\n StateOfJS\n
`state-of-javascript-2018`	State of Javascript Annual Survey 2018	\n StateOfJS\n
`state-of-javascript-2019`	State of Javascript Annual Survey 2019	\n StateOfJS\n
`countries-languages-spoken`	Languages Spoken in Different Countries	\n Infoplease\n

\n\nMore datasets will be added soon..\n\n## Contributing\n\nThis is an open source project and we welcome contributions.\n\n### Local Development Setup\n\n1. Clone the repository:\n\n```\ngit clone https://github.com/JovianML/opendatasets.git\n```\n\n2. Setup the Python environment for development\n\n```\nconda create -n opendatasets python=3.5\nconda activate opendatasets\npip install -r requirements.txt\n```\n\n3. Open up the project in VS code and make your changes. Make sure to install the Python Extension for VS Code and select the `opendatasets` conda environment.\n\nThis package is developed and maintained by the [Jovian](https://www.jovian.ai) team.\n\n\n","description_content_type":"text/markdown","docs_url":null,"download_url":"","downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":"https://github.com/JovianML/opendatasets","keywords":"","license":"","license_expression":null,"license_files":null,"maintainer":"","maintainer_email":"","name":"opendatasets","package_url":"https://pypi.org/project/opendatasets/","platform":null,"project_url":"https://pypi.org/project/opendatasets/","project_urls":{"Homepage":"https://github.com/JovianML/opendatasets"},"provides_extra":null,"release_url":"https://pypi.org/project/opendatasets/0.1.22/","requires_dist":["tqdm","kaggle","click"],"requires_python":"","summary":"A curated collection of datasets for data analysis & machine learning, downloadable with a single Python command","version":"0.1.22","yanked":false,"yanked_reason":null}

Dependencies of opendatasets (3).
Dependency	Optional
click	false
kaggle	false
tqdm	false

Package search

opendatasets

project_urls

Dependencies

All package metadata

Releases