AID302 Optimization for Data Science
AID302 Optimization for Data Science
Syllabus | International University of Sarajevo - Last Update on Feb 02, 2026
Syllabus Quick Jump
Search and navigate to any syllabus instantly
Artificial Intelligence and Data Engineering
Seyednima Rabiei
Course Lecturer
Course Objectives
This course provides a comprehensive exploration of mathematical optimization and its applications in data science. Students will receive an introduction to the fundamental principles of mathematical optimization, including line search methods, gradient-based methods, Newton's method, Hessian-based methods, and derivative-free optimization. The course also covers techniques for solving linear and nonlinear optimization problems, with a specific focus on least squares techniques. Throughout the course, students will gain hands-on experience in applying optimization algorithms to real-world data science applications. By the end of the course, students will have a solid understanding of mathematical optimization and its practical relevance in data-driven decision-making.
Learning Outcomes
After successful completion of the course, the student will be able to:
Course Materials
Required Textbook
Optimization for Data Analysis by STEPHEN J. WRIGHT, BENJAMIN RECHT
Additional Literature
1. Numerical Optimization by Jorge Nocedal and Stephen J. 2. Convex Optimization by Stephen Boyd and Lieven Vandenberghe. 3.Optimization for Data Analysis by STEPHEN J. WRIGHT, BENJAMIN RECHT.Teaching Methods
Weekly Topics
| Week | Topic | Readings / References |
|---|---|---|
| 1 | Introduction to Optimization: What is optimization? Examples from engineering, ML, economics, Classification: Linear vs Nonlinear, Constrained vs Unconstrained, Convex vs Nonconvex | |
| 2 | Mathematical Foundations: Gradients and Hessians, Taylor expansion, First-order and second-order optimality conditions | |
| 3 | Convex Sets and Convex Functions: First-order condition for convexity, Second-order condition, Examples | |
| 4 | Convex Sets and Convex Functions: First-order condition for convexity, Second-order condition, Examples | |
| 5 | Steepest descent method and its convergence analysis in the general case, the convex case and the strongly convex case. | |
| 6 | Unconstrained Optimization – First Order Methods: Gradient descent, Step size selection: Exact line search, Backtracking, Armijo rule, Convergence analysis, Programming assignment: Implement gradient descent. | |
| 7 | Exam | |
| 8 | Unconstrained Optimization – First Order Methods: Gradient descent, Step size selection: Exact line search, Backtracking, Armijo rule, Convergence analysis, Programming assignment: Implement gradient descent. | |
| 9 | Second Order Methods: Newton’s method, Modified Newton, Quasi-Newton Methods, Secant method (1D), BFGS method, DFP method, Limited-memory BFGS (conceptual) | |
| 10 | Second Order Methods: Newton’s method, Modified Newton, Quasi-Newton Methods, Secant method (1D), BFGS method, DFP method, Limited-memory BFGS (conceptual) | |
| 11 | Constrained Optimization: Equality constraints, Lagrange multipliers, KKT conditions, Geometric interpretation | |
| 12 | Constrained Optimization: Equality constraints, Lagrange multipliers, KKT conditions, Geometric interpretation | |
| 13 | Stochastic Gradient Descent (SGD), Batch vs stochastic methods, Mini-batch SGD Learning rate schedules, Convergence intuition | |
| 14 | Stochastic Gradient Descent (SGD), Batch vs stochastic methods, Mini-batch SGD Learning rate schedules, Convergence intuition | |
| 15 | Accelerated Methods: Momentum method |
Course Schedule (All Sections)
| Section | Type | Day 1 | Venue 1 | Day 2 | Venue 2 |
|---|---|---|---|---|---|
| AID302.1 | Course | Wednesday 14:00 - 16:50 | A F1.4 - Class/Laboratory | - | - |
| AID302.1 | Tutorial | Thursday 15:00 - 16:50 | B F1.17 | - | - |
Office Hours & Room
| Day | Time | Office | Notes |
|---|---|---|---|
| Monday | 10:00 - 11:30 | A F2.4 | |
| Tuesday | 13:00 - 15:00 | A F2.4 | |
| Wednesday | 13:00 - 15:00 | A F2.4 | |
| Thursday | 10:00 - 13:00 | A F2.4 | |
| Friday | 10:00 - 12:00 | A F2.4 |
Assessment Methods and Criteria
Assessment Components
Final Exam
AI: Not AllowedAlignment with Learning Outcomes : 1 2 3 4
Midterm
AI: Not AllowedAlignment with Learning Outcomes : 1 2 3 4
IUS Grading System
| Grading Scale | IUS Grading System | IUS Coeff. | Letter (B&H) | Numerical (B&H) |
|---|---|---|---|---|
| 0 - 44 | F | 0 | F | 5 |
| 45 - 54 | E | 1 | ||
| 55 - 64 | C | 2 | E | 6 |
| 65 - 69 | C+ | 2.3 | D | 7 |
| 70 -74 | B- | 2.7 | ||
| 75 - 79 | B | 3 | C | 8 |
| 80 - 84 | B+ | 3.3 | ||
| 85 - 94 | A- | 3.7 | B | 9 |
| 95 - 100 | A | 4 | A | 10 |
Late Work Policy
Information about late submission policies will be shared during class and posted in this section. Please check back for official guidelines.
ECTS Credit Calculation
📚 Student Workload
This 6 ECTS credit course corresponds to 150 hours of total student workload, distributed as follows:
Lecture hours
42 hours ⏳ (14 week × 3 h)
Assignments
21 hours ⏳ (7 week × 3 h)
Active labs
28 hours ⏳ (14 week × 2 h)
Home study
14 hours ⏳ (14 week × 1 h)
In-term exam study
10 hours ⏳ (1 week × 10 h)
Final exam study
11 hours ⏳ (1 week × 11 h)
Term project/presentation
24 hours ⏳ (12 week × 2 h)
150 Total Workload Hours
6 ECTS Credits
Course Policies
Academic Integrity
All work submitted must be your own. Plagiarism, cheating, or any form of academic dishonesty will result in disciplinary action according to university policies. When in doubt about citation practices, consult the instructor.
Attendance Policy
Students are expected to adhere to the attendance requirements as outlined in the International University of Sarajevo Study Rules and Regulations. Excessive absences, whether excused or unexcused, may impact academic performance and eligibility for assessment. Mandatory sessions (e.g., labs, workshops) require attendance unless formally exempted. For detailed policies on absences, documentation, and penalties, please refer to the official university regulations.
Technology & AI Policy
Laptops/tablets may be used for note-taking only during lectures. Phones should be silenced and put away during all class sessions. Audio/video recording requires prior permission from the instructor.
Artificial Intelligence (AI) Usage: The use of AI tools (e.g., ChatGPT, Copilot, Gemini) varies by assessment component. Please refer to the AI usage indicator next to each assessment item in the Assessment Methods and Criteria section above. Submitting AI-generated content as your own work, where AI is not explicitly allowed, constitutes an academic integrity violation.
Communication Policy
All course-related communication should occur through official university channels (institutional email or SIS). Emails should include [AID302] in the subject line.
Academic Quality Assurance Policy
Course Academic Quality Assurance is achieved through Semester Student Survey. At the end of each academic year, the institution of higher education is obliged to evaluate work of the academic staff, or the success of realization of the curricula.
Learning Tips
Be prepared to contribute thoughtfully during class discussions, labs, or collaborative work. Active participation deepens understanding and encourages critical thinking.
Complete assigned readings or prep materials before class. Take notes, highlight key ideas, and jot down questions. Aim to grasp core concepts and their applications—not just facts.
Use course frameworks or methodologies to analyze problems, case studies, or projects. Begin early to allow time for reflection and refinement. Seek feedback to improve your work.
Don’t hesitate to reach out when something is unclear. Use office hours, discussion boards, or peer networks to clarify concepts and stay on track.
Syllabus Last Updated on Feb 02, 2026 | International University of Sarajevo
Print Syllabus
Referencing Curricula Print this page
| Course Code | Course Title | Weekly Hours* | ECTS | Weekly Class Schedule | ||||||
| T | P | |||||||||
| AID302 | Optimization for Data Science | 3 | 2 | 6 | ||||||
| Prerequisite | None | It is a prerequisite to | - | |||||||
| Lecturer | Seyednima Rabiei | Office Hours / Room / Phone | Monday: 10:00-11:30 Tuesday: 13:00-15:00 Wednesday: 13:00-15:00 Thursday: 10:00-13:00 Friday: 10:00-12:00 |
|||||||
| nrabiei@ius.edu.ba | ||||||||||
| Assistant | Assistant E-mail | |||||||||
| Course Objectives | This course provides a comprehensive exploration of mathematical optimization and its applications in data science. Students will receive an introduction to the fundamental principles of mathematical optimization, including line search methods, gradient-based methods, Newton's method, Hessian-based methods, and derivative-free optimization. The course also covers techniques for solving linear and nonlinear optimization problems, with a specific focus on least squares techniques. Throughout the course, students will gain hands-on experience in applying optimization algorithms to real-world data science applications. By the end of the course, students will have a solid understanding of mathematical optimization and its practical relevance in data-driven decision-making. | |||||||||
| Textbook | Optimization for Data Analysis by STEPHEN J. WRIGHT, BENJAMIN RECHT | |||||||||
| Additional Literature |
|
|||||||||
| Learning Outcomes | After successful completion of the course, the student will be able to: | |||||||||
|
||||||||||
| Teaching Methods | ||||||||||
| Teaching Method Delivery | Face-to-face | Teaching Method Delivery Notes | ||||||||
| WEEK | TOPIC | REFERENCE | ||||||||
| Week 1 | Introduction to Optimization: What is optimization? Examples from engineering, ML, economics, Classification: Linear vs Nonlinear, Constrained vs Unconstrained, Convex vs Nonconvex | |||||||||
| Week 2 | Mathematical Foundations: Gradients and Hessians, Taylor expansion, First-order and second-order optimality conditions | |||||||||
| Week 3 | Convex Sets and Convex Functions: First-order condition for convexity, Second-order condition, Examples | |||||||||
| Week 4 | Convex Sets and Convex Functions: First-order condition for convexity, Second-order condition, Examples | |||||||||
| Week 5 | Steepest descent method and its convergence analysis in the general case, the convex case and the strongly convex case. | |||||||||
| Week 6 | Unconstrained Optimization – First Order Methods: Gradient descent, Step size selection: Exact line search, Backtracking, Armijo rule, Convergence analysis, Programming assignment: Implement gradient descent. | |||||||||
| Week 7 | Exam | |||||||||
| Week 8 | Unconstrained Optimization – First Order Methods: Gradient descent, Step size selection: Exact line search, Backtracking, Armijo rule, Convergence analysis, Programming assignment: Implement gradient descent. | |||||||||
| Week 9 | Second Order Methods: Newton’s method, Modified Newton, Quasi-Newton Methods, Secant method (1D), BFGS method, DFP method, Limited-memory BFGS (conceptual) | |||||||||
| Week 10 | Second Order Methods: Newton’s method, Modified Newton, Quasi-Newton Methods, Secant method (1D), BFGS method, DFP method, Limited-memory BFGS (conceptual) | |||||||||
| Week 11 | Constrained Optimization: Equality constraints, Lagrange multipliers, KKT conditions, Geometric interpretation | |||||||||
| Week 12 | Constrained Optimization: Equality constraints, Lagrange multipliers, KKT conditions, Geometric interpretation | |||||||||
| Week 13 | Stochastic Gradient Descent (SGD), Batch vs stochastic methods, Mini-batch SGD Learning rate schedules, Convergence intuition | |||||||||
| Week 14 | Stochastic Gradient Descent (SGD), Batch vs stochastic methods, Mini-batch SGD Learning rate schedules, Convergence intuition | |||||||||
| Week 15 | Accelerated Methods: Momentum method | |||||||||
| Assessment Methods and Criteria | Evaluation Tool | Quantity | Weight | Alignment with LOs | AI Usage |
| Final Exam | 1 | 40 | 1,2,3,4 | Not Allowed | |
| Semester Evaluation Components | |||||
| Midterm | 1 | 60 | 1,2,3,4 | Not Allowed | |
| *** ECTS Credit Calculation *** | |||||
| Activity | Hours | Weeks | Student Workload Hours | Activity | Hours | Weeks | Student Workload Hours | |||
| Lecture hours | 3 | 14 | 42 | Assignments | 3 | 7 | 21 | |||
| Active labs | 2 | 14 | 28 | Home study | 1 | 14 | 14 | |||
| In-term exam study | 10 | 1 | 10 | Final exam study | 11 | 1 | 11 | |||
| Term project/presentation | 2 | 12 | 24 | |||||||
| Total Workload Hours = | 150 | |||||||||
| *T= Teaching, P= Practice | ECTS Credit = | 6 | ||||||||
| Course Academic Quality Assurance: Semester Student Survey | Last Update Date: 22/02/2026 | |||||||||