Introduce students to big data management and associated issues. The topics include overview of the Apache toolset (Hadoop, Spark, and others), distributed file systems, big data programming models, data warehousing and big data security and protection; visualization tools and frameworks. (Note: module to be offered on MScSE (Data Science option), MScDA (KBS' Data Analytics) and ME (Computer Engineering)

1. "Big data": meaning and sources; the Vs of big data; data governance: accuracy, availability, usability and security; impacts of big data, industrial and societal. 2. Big data programming frameworks and systems: distributed file systems, scalable computing, the MapReduce programming model, the Spark programming and computing model, overview of the main components of the Hadoop ecosystem. 3. Data warehousing concepts: what is a data warehouse; role of a data warehouse in data management; architecture of a data warehouse; ETL: extraction, transformation, load process, data marts; operational systems vs. data warehouses. 4. Big data security and protection challenges and practices, such as privacy-preserving data composition, encryption, granular access control, user authentication models, endpoint filtering and validation, etc. 5. Relational information in a business context; visualization challenges; graph / network visualization frameworks: Sugiyama and force-directed layout methods.

1. Recognise the technological challenges in big data governance. 2. Summarise the Hadoop ecosystem. 3. Describe the MapReduce programming model. 4. Describe the architectural components of a data warehouse. 5. Discuss the big data security challenges and practices.

1. Discuss the impact of big data on industry and society. 2. Recognise the importance of big data security. 3. Awareness of ethical issues associated with use and misuse of big data.

N/A

The module is taught in the form of lectures and lab practice in a computer lab with state-of-the-art software for big data management.