or  Google Search
Department of Statistical and Actuarial Sciences
Center of Actuarial Excellence (CAE)

Markov Chains with Applications

Statistical Science 4654A Section 001

Course Outline

Make PDF
Instructor Information
InstructorDavid Stanford
OfficeWSC 211
Phone519-661-3612 ext 83612
Office Hours Tue,Wed 4:00-4:25; Tue,Wed 5:25-5:55;
Course Information
Course DescriptionContinuous-time Markov chains, applications to phase-type distributions, Markov chain Monte Carlo simulation and queuing theory.
PrerequisitesSS3657A/B A minimum mark of 60% in Statistical Sciences 3657A/B. A minimum mark of 60% in Statistical Sciences 3657A/B.
Antirequisites The former Statistical Sciences 3652A/B, former Statistical Sciences 4652A/B, former Statistical Sciences 4657A/B and former Statistical Sciences 4737A/B.
Pre, Co-requisites WarningUnless you have either the requisites for this course or written special permission from your Dean to enroll in it, you may be removed from this course and it will be deleted from your record. You can be deregistered at any time even after writing the final exam. This decision may not be appealed. You will receive no adjustment to your fees in the event that you are dropped from a course for failing to have the necessary prerequisites
Lecture Hours Mon,Tue,Wed 4:30 PM-5:30 PM WSC 240; 
Book Information
Text Book(s)Introduction to Probability Models, 11th edition, by Sheldon M. Ross (Academic Press)
Course Objectives

Following a brief review of discrete time Markov chains, this course provides a thorough treatment of the theory and application of continuous time Markov chains, including phase-type distributions. Students will become familiar with Markov Chain Monte Carlo methods. A substantial part of the course pertains to the application of both discrete-time and continuous-time Markov chains to model queues. In addition to the theory of queues, students will gain insight regarding how congestion manifests itself in single-server and multi-server queues under first-come, first-served (FCFS) and priority arrangements. Networks of queues will be considered as well. The role of occupancy and variability on the degree of congestion will be analysed, as well as the impacts of pooling of service resources and priority queueing arrangements. Motivating examples will be drawn from call centres and health care applications.

A rough timeline of material to be covered is as follows:

Review of discrete time Markov chains (Chapter 4), strong and weak convergence (2 weeks)

Markov chain Monte Carlo methods (section 4.9 and supplements) (1 week)

Continuous time Markov chains (Chapter 6), including Phase-type distributions (5 weeks)

Theory and application of queues (Chapter 8 and supplemental notes) (5 weeks)
Assignments and/or Quizzes

There will be four or five assignments at roughly two-week intervals, worth seven percent of the final grade. In addition, there will be two class participation components. Firstly, there will be  hands-on sessions in the Western Active Learning Space (WALS), UCC 66, on four dates to be determined. WALS sessions are worth a total of four percent of the final grade. 

Secondly, the classes will be run with an active-participation component worth four percent as well.



Midterms or Tests
In class on September 11th, it will be decided whether to have a single two-hour midterm test roughly midway through the course (LATER IN OCTOBER), or two one-hour tests at roughly monthly intervals. Majority rule will decide this. In total, midterm evaluation will be worth 35% of the final grade.
Final Exam
The date and time for the 3-hour final exam will be announced by the registrar's office, usually near the first part of November. Students are required to wait until the final exam schedule is posted before making arrangements for travel over the Christmas period. Exams will not be moved to accommodate students who ignore this requirement.
The final grade weights are as follows: 50% final exam, 35% for the midterm exam, 7% for assignments, 4% for  group activities in the WALS classroom, and 4% for classroom participation.
Course Website Information

All assignments and lecture notes will be posted on this website.


Computing and/or Calculator Requirement
Non-programmable calculators only for tests and the final exam. The best choice is from among those acceptable for Society of Actuaries exams. See https://www.soa.org/education/exam-req/exam-day-info/edu-calculators.aspx
Active classroom and office-hour participation
The most active participant(s) in classroom and office-hour consults will receive four percent towards the final grade, and all others will receive a score in proportion to the ratio of the logarithms of their number of valued contributions to that of the most active participant. (Ex: A student who answers 5 questions will receive half the participation marks of one answering 25 questions. Everyone is credited with 1 contribution, so that the logarithm calculations will be non-negative values.). NB MERELY ATTENDING CLASSES DOES NOT CONSTITUTE PARTICIPATION!
Western Adaptive Learning Space (WALS) use, Room UCC 66
There will be four WALS sessions at roughly monthly intervals from September through December. Your active presence is required to the mark for each session (four percent in total).
Department Policy on Missed Course Requirements and Student Health and Wellness

If you are unable to meet a course requirement due to illness or other serious circumstances, you must provide valid medical or other supporting documentation to your Dean's office as soon as possible, and contact your instructor immediately. If accommodation is approved by your Dean's office, your instructor will be notified, then it is your responsibility to make alternative arrangements with your instructor. In the event of a missed final exam, a "Recommendation of Special Examination" form must be obtained from the Dean's Office immediately. For further information please see: http://www.stats.uwo.ca/accommodation_medical.pdf. A student requiring academic accommodation due to illness, should use the Student Medical Certificate when visiting an off-campus medical facility. The form can be found at: http://www.stats.uwo.ca/medicalform.pdf. Or, request a Record's Release Form (located in the Dean's Office) for visits to Student Health Services.

Missed Midterm or Test: The policy of the department of Statistical and Actuarial Sciences is that there will be no make-up exams for a missed midterm. For those that do legitimately miss a midterm and provide the required supporting documentation, the standard practice will be that the weight of the midterm will be reassigned to the final exam. If your reason is not deemed valid, then you will receive a mark of 0.

Student Health and Wellness: As part of a successful student experience at Western, we encourage students to make their health and wellness a priority. Western provides several on campus health-related services to help you achieve optimum health and engage in healthy living while pursuing your degree. For example, to support physical activity, all students, as part of their registration, receive membership in Western’s Campus Recreation Centre. Numerous cultural events are offered throughout the year. Please check out the Faculty of Music web page http://www.music.uwo.ca/, and our own McIntosh Gallery http://www.mcintoshgallery.ca. Information regarding health- and wellness-related services available to students may be found at http://www.health.uwo.ca/.
Students seeking help regarding mental health concerns are advised to speak to someone they feel comfortable confiding in, such as their faculty supervisor, their program director (graduate chair), or other relevant administrators in their unit. Campus mental health resources may be found at http://www.health.uwo.ca/mental_health/resources.html.
To help you learn more about mental health, Western has developed an interactive mental health learning module, found here: http://www.health.uwo.ca/mental_health/module.html. This module is 30 minutes in length and provides participants with a basic understanding of mental health issues and of available campus and community resources. Topics include stress, anxiety, depression, suicide and eating disorders. After successful completion of the module, participants receive a certificate confirming their participation.

Accessibility: Please contact the course instructor if you require lecture or printed material in an alternate format or if any other arrangements can make this course more accessible to you. You may also wish to contact Services for Students with Disabilities (SSD) at 661-2111 ext. 82147 if you have questions regarding accommodation.

Support Services: Learning-skills counsellors at the Student Development Centre (SDC) (http://www.sdc.uwo.ca) are ready to help you improve your learning skills. They offer presentations on strategies for improving time management, multiple-choice exam preparation/writing, textbook reading, and more. Individual support is offered throughout the Fall/Winter terms in the drop-in Learning Help Centre, and year-round through individual counselling.
Additional student-run support services are offered by the USC, http://westernusc.ca/services.
The website for Registrarial Services is http://www.registrar.uwo.ca.

Email Communication
You are welcome to communicate with me by email, but E-mail communication should only be used to provide me with information or to ask a question that requires a brief response. For more lengthy discussions and for discussions on lectures/course material please see me during their scheduled office hours or by appointment. If you do email them, please use your UWO account, as these are often the only emails read (as emails sent from other addresses often get spammed).
Attendance: Classroom attendance is viewed as an important part of the learning process. Students are advised that excessive absenteeism may result in the student being disbarred from the final exam (see Western Academic Calendar). Class Environment Policy: The Department has adopted a "Mutual Expectations" policy governing the classroom environment and all work submitted by students. The full text of the policy can be found at: http://www.stats.uwo.ca/mutual_expectations.pdf. In summary, the policy was developed under the premise that all interactions between students and faculty should be governed by the principles of courtesy, respect and honesty. Students who are in emotional/mental distress should refer to Mental Health@Western http://www.uwo.ca/uwocom/mentalhealth/ for a complete list of options about how to obtain help. Academic Integrity: We expect our students to conduct themselves academically in a manner that upholds the integrity and reputation of our academic programs. Cheating on assignments, exams, essays and term papers is considered to be a serious violation of ethical conduct, and will not be tolerated.
Western Links
Registrarial Services: http://http://www.registrar.uwo.ca/ Learning-skills services at the SDC: http://http://www.sdc.uwo.ca/ Universities Students' Council: http://westernusc.ca/