Welcome to your third year and the Information Engineering module! In this semester, we will get to know information systems and information engineering more closely.
Industry, commerce and research are being transformed by the potential to capture, store, manipulate, analyse and visualise data and information on a massive scale. Data-driven discovery and prediction are yielding new insights to businesses and academe in every domain, while information retrieval in the form of web search engines are the default mechanism by which most individuals prefer to find information.
Information Engineering is the engineering discipline that deals with the generation, distribution, analysis, and use of information, data, and knowledge in systems.
The components of information engineering include more theoretical fields such as machine learning, artificial intelligence and information theory, and more applied fields such as computer vision, natural language processing, bioinformatics, medical image computing, and robotics. Many of these originate from computer science, as well as other branches of engineering such as computer engineering, electrical engineering, and bioengineering.
"Big Data" in all its varied forms requires a set of statistical, analytical and programming skills, which are currently in short supply, and likely to remain so for the foreseeable future as this remains a bleeding edge growth area in terms of career opportunities. This module aims to provide students with the core skill set for information engineering, drawing across data science, artificial intelligence, operational research, information retrieval, and related technologies.
Join me in this module this semester where we will learn the basic principles and practice of information, information systems and develop the skills to generate, maintain and extract insights from data. Topics we will cover will include:
- History and context of data science and information engineering
- Technology landscape
- Parallel databases and query processing, and in-database analytics
- MapReduce and Hadoop
- Key-value stores and NoSQL, examples of NoSQL implementations
- Programming using Python and R
- Topics in information retrieval
- Topics in statistical modelling: statistics review, concepts, design, pitfalls
- Topics in machine learning: supervised learning, optimisation, unsupervised learning, classification, mining, forecasting
- Topics in reasoning with uncertainty
- Topics in visualisation and graph analytics
- Issues in privacy and ethics
By the end of this module, you should be able to:
- develop knowledge and understanding of the statistical, algorithmic, and analytical concepts and methods which underpin and embody the core of data science and information engineering
- develop the skills and intuitions necessary to select and design an information engineering solution to practical data and information problems
- develop skills to successfully apply statistical, algorithmic, and analytical techniques to implement an information engineering solution using appropriate programming languages and software packages and draw rational conclusions based on the results of the solution
By the end of this module, you will be able to:
- Demonstrate a systematic understanding and knowledge of the statistical, algorithmic and analytical concepts and methods which form the core of data science and information engineering
- Identify the most appropriate techniques and tools for a particular data or information problem, and be critically aware of the limitations on these methods
- Design an information engineering solution to a particular data or information problem from a toolbox of appropriate techniques
- Implement an information engineering solution, including such aspects as data modelling at scale, statistical modelling, programming and visualisation of results
- Draw relevant and rational conclusions from the application of an information engineering solution to a particular data or information problem, and critically appreciate the uncertainties and limitations to the accuracy and/or applicability of these results
| Session | Topic | Lecture | Practical/Lab | Additional Digital Self-paced Training | Knowledge check |
|---|---|---|---|---|---|
| Lecture 1 | Information Engineering Concepts, Data Driven Organisations | Module Overview L1 slides |
Activity 1-1: Reflecting on the Importance of an Information Management System Activity 1-2: Reflecting on information |
||
| Lecture 2 | |||||
| Lecture 3 | |||||
| Lecture 4 | |||||
| Lecture 5 | |||||
| Lecture 6 | |||||
| Lecture 7 | |||||
| Lecture 8 | |||||
| Lecture 9 | |||||
| Lecture 10 | |||||
| Lecture 11 | |||||
| Lecture 12 |
Information Engineering Concepts
Details
Welcome to the Information Engineering module 👋.
An overview of the module aims and objectives is available under Content > Module Information. Please make sure you have a look at these pages before you proceed with the rest of the pages on this module. Make sure you have spotted the Readings page and that you have glanced through the other resources mentioned there.
We start this module by looking at the basics of information engineering and information systems engineering, including the history and the context of data science and information engineering and how it is being shaped by and has shaped the era of AI (and more recently Generative AI).
We will also be looking at what information, data and information engineering is, when they were first discussed in the computing world and their importance for businesses and society. We will also be discussing different ways information is understood across different fields of databases that you might come across (relational, non-relational, graph etc). Finally, we will be working in groups to discuss related issues and exchange ideas and experiences.
In the practical/lab, we will be refreshing our memory of programming basics and trying out some examples of information pipelines using open datasets and Python code in kaggle.com.
For the assessment on this module you will need to work on a fictional scenario of an organisation presented to you where you will need to identify and propose ways it could build or improve its data engineering architecture and data analytics pipelines. Assessment information will be made available under Content > Module Assessment after Lecture 2. Don’t worry about what that will involve just yet. We will get there!
You should have everything you need to make a start on the assessment after Lecture 6 content is covered.
Practicals/Labs
All lectures are followed by an AWS Academy (or other cloud provider) training set of material - please make sure you gain access to AWS Academy training material (I will be providing details during class about how I will be adding you to these), and follow through the assigned module material (videos, labs and knowledge checks) to reinforce your knowledge of the contents covered.
Readings for this Lecture
Detailed information for today's key concepts can be found in the following readings:
- 📘 Reading 1-1: Chapter 1 of Reis and Housley (2022)
- 📘 Reading 1-2: Chapter 2 of Reis and Housley (2022)
- 📘 Reading 1-3: Chapter 1 of Heijden (2009)
- 📘 Video Resource 1-4: What is information theory?
- 📘 Video Resource 1-5: IBM Cloud - what is data science?
- 📘 Video Resource 1-6: The Science of Information - a lecture by Professor Richard Harvey (23/10/2018)
Case studies:
- Stripe: AWS re:Invent 2022 - Modernize your data archive with Amazon S3, featuring Stripe (STG205) (youtube.com)
- AWS re:Invent 2022 - Achieving your modern data architecture (ANT205-R)
Labs:
- Lab 1.1: Querying data using Athena (AWS Data Engineering > Module 04)
<iframe src="https://uos.cloud.panopto.eu/Panopto/Pages/Embed.aspx?id=d9bcd8f7-6344-493d-b90b-b1fc008edfe7&autoplay=false&offerviewer=true&showtitle=true&showbrand=true&captions=false&interactivity=all" height="405" width="100%" style="border: 1px solid #464646;" allowfullscreen="allowfullscreen" allow="autoplay"></iframe>
<iframe src="https://uos.cloud.panopto.eu/Panopto/Pages/Embed.aspx?id=035acb00-ed50-4baf-9032-b1fc00a2116a&autoplay=false&offerviewer=true&showtitle=true&showbrand=true&captions=false&interactivity=all" height="405" width="100%" style="border: 1px solid #464646;" allowfullscreen="allowfullscreen" allow="autoplay"></iframe>
| Module Title | Cloud Computing for Data Science and AI |
|---|---|
| Module Code | IPLDSAM03 (apprentices) / IMDSAM03 (mainstream) |
| Level | 6 (3rd year undergraduate) |
| School | School of Technology, Business and Arts |
| Github Repo | https://github.com/KakiasAcademicAlcove/InformationEngineering/ |
