Overview

Tackling big data is a roadblock that stops many promising machine learning projects from going into production. Data engineering is a complex task that makes raw data usable for analysis and predictive modeling. This topic will focus on the practical challenges for maintaining the reliability of data pipelines, e.g.,  data … For more content click the Read More button below.

Topic availabilities

To view topic availabilities, select an availability from the drop down, towards the top right of the screen.

Tuition pattern

To view tuition patterns, select an availability from the drop down, towards the top right of the screen.

Aims

This topic aims to provide:

  • Skills to collect structured and unstructured data using APIs and web wrangling 
  • Skills to store, manage and pre-process data and perform exploratory data analysis 
  • Methodological skills for analysing data for predictive modelling such as those from machine learning and statistical data analysis 
  • Learn ethical data collection and data manipulation  
  • Skills to analyse results from big data to get actionable insights using sophisticated data visualisation methods 

Learning outcomes

On completion of this topic you will be expected to be able to:
1.
Collect structured and unstructured data using APIs and web wrangling
2.
Pre-process data and generate statistics
3.
Store and manage data in files and nosql databases
4.
Apply appropriate models on big data for predictive modelling and forecasting
5.
Reflect on the ethical challenges in data collection and data manipulation
6.
Work independently or in a team to get actionable insights using sophisticated data visualisation methods
7.
Present your results and methods to a wide audience

Assessments

To view assessment information, select an availability from the drop down, towards the top right of the screen.

Requisites information

Pre-requisites:
Anti-requisites:

Assumed knowledge

Basic Knowledge of Statistics; Sound knowledge of at least one programming/scripting language, e.g., C++/Java/Matlab/R/Python such as can be obtained by completing COMP1711.