Depending on the data source, this turns out to be more complicated than one might guess. The first part is about getting publicly available weather data and about extracting relevant metrics from this data. Getting the data (you are just reading this part).Since the whole analysis from downloading the data until performing some analysis involves many steps, the whole journey is split up into three separate articles as follows You can find a Jupyter notebook containing the complete working code on GitHub. Many details of processing steps are omitted in this article to keep focus on the general approach. The purpose of the article series actually is two-fold: Diving into working with weather measurements and providing a non-trivial example for using PySpark with real data. In this article series I want to present my approach of using PySpark for analyzing ca 100GB of compressed raw weather data for reconstructing some relevant metrics substantiating the climate change. Depending on the chosen data source, following this idea can be a technically challenging and insightful journey into weather data. While I am by no means an expert for climate or weather, I was wondering if I could follow the claims of an increase of the average temperature by analyzing appropriate data. Nevertheless some people don’t believe these experts and claim that the climate didn’t change, and other people question the influence of the human species on the current development. The climate change currently is a hot topic, with many experts claiming a significant increase of the average temperature over the whole world.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |