The goal

This training is devoted to both developers and system engineers. Our goal is to learn techniques used with large scaling and highly available systems. We will study the example of building web application.

We’ll discuss:

  • horizontal scaling techniques,
  • load-balancing,
  • redundancy protocols,
  • session synchronization,
  • database distribution,
  • cloud architectures,
  • caching,
  • workload queuing.

Exam rules

  • we’ll have one or two parts of the exam
  • final result will be recalculated to the maximum number of points equal to 100;
  • the final mark points range:
    • do 50 pkt – 2,
    • 50-59,9 pkt – 3,
    • 60-69,9 pkt –  3,5,
    • 70-79,9 pkt – 4,
    • 80-89,9 pkt – 4,5
    • 90-100 pkt – ocena 5,

Laboratories rules

  • we’ll have two or three projects,
  • each project will be marked with 0-100 points,
  • the final result will be proportionally calculated so the range will be from 0 to 100;
  • the final mark points range:
    • do 50 pkt – 2,
    • 50-59,9 pkt – 3,
    • 60-69,9 pkt –  3,5,
    • 70-79,9 pkt – 4,
    • 80-89,9 pkt – 4,5
    • 90-100 pkt – ocena 5,

Project #2

Functional requirements:

  • Account creation (profile with Name, Surname, Address, NIP,PESEL, avatar – uploaded with standarized size) – 5 points.
  • Sessions in Redis – 25 points
  • Wall is similar to Facebook wall, but we have only one wall for all users, just with text (max 256 chars) and it shows only last 10 posts. The current content of wall should be cached using Redis. After every update of the wall the cache should be updated. – 15 points
  • When a user sends some text to a wall, it will be added to a queue (RabbitMQ) for admin approval. Admin can check all the posts waiting in queue and can either approve (meaning the post will go to the wall) or reject it (the post will be discarded) – 15 points

Cluster requirements:

  • The app should run on both web server RS1 and RS2, load-balanced with LVS using Keepalived – 10 points.
  • Each App server should connect to Mysql Master for necessary queries (like updates, but also some selects), and to slave for other queries – 20 points.
  • Gluster should cover the availability of avatar on all server – 10 points.
  • The architecture design should look like this.

Caution:

Having a working code doesn’t mean you’ll get any points. To each and every function the teacher can ask some questions (or give you some test) related to the subject and only both the correct answers and working code will be rewarded with points.

General rules

  1. Distributed Operating Systems is one semester course.
  2. It means we have 15 labs and 15 lectures.
  3. The original (faculty) schedule planned our classes to Wednesday 13:15-15:15 and 15:30-17:00. We changed it though on a request from students 🙂
  4. First two meeting were held on Feb 22nd and March 1st. We then decided to meet on to two weekends. First is on April 8th and 9th.

Bibliography

  • Linux Virtual Server High Availability using VRRPv2, Alexandre Cassen, Linux Virtual Server OpenSource Project, Paris, France,
  • LVS-HOWTO, Joseph Mack, v2009.09 Sep 2009, released under GPL,
  • The NIST Definition of Cloud Computing, Recommendations of the National Institute of Standards and Technology, Special Publication 800-145, Peter Mell, Timothy Grance,
  • Distributed Operating Systems, A. Tanenbaum, 1994,

Online

Exam

Date and time options:

  • Tuesaday, June 6th, 2017, 13:45-20:00
  • Wednesday, June 7th, 2017, 13:45-20:00
  • Wednesday, June 21st, 2017, 8:00-11:30

The Schedule

Zajęcia

Temat zajęć

Data

 1.

Lecture:

  • Distributed Operating System Basics

Labs:

  • Environment preparation

Lecture:

  • 22.02.2017, 13:45-15:15

Labs:

  • 22.02.2017, 15:30-17:00
 2. Lecture:

  • Simple methods of load balancing and workload distribution

Labs:

  • LVS – VM configuration (daemons, network, php, mysql, etc)

Lecture:

  • 1.03.2017, 13:45-15:15

Labs:

  • 1.03.2017, 15:30-17:00
 3.

Lecture:

  • LVS-NAT, LVS-DR

Labs:

Lecture:

  • 7.04.2017, 8:40-10:00

Labs:

  • 7.04.2017, 10:05-11:25
 4. Lecture:

  • LVS-TUN, Dynamic Routing Method

Labs:

Lecture:

  • 7.04.2017, 11:30-12:50

Labs:

  • 7.04.2017, 13:20-14:40
 5.

Lecture:

  • Virtual Routing Redundancy Protocol

Labs:

Lecture:

  • 7.04.2017, 15:00-16:20

Labs:

  • 7.04.2017, 16:30-18:00
6.

Lecture:

  • The LB, HA architecture design

Labs:

Lecture:

  • 8.04.2017, 10:05-11:25

Labs:

  • 8.04.2017, 13:20-14:40
 7. Lecture:

  • Caching (reverse proxy, key-value)
  • Workload queuing

Labs:

Lecture:

  • 8.04.2017, 11:30-12:50

Labs:

  • 8.04.2017, 15:00-16:20
 8. Lecture:

  • Database scaling

Labs:

Lecture:

  • 8.04.2017, 8:40-10:00

Labs:

  • 8.04.2017, 16:30-18:00
 9.

Lecture:

  • Introduction to sessions synchronization

Labs:

Lecture:

  • 20.05.2017, 8:40-10:00

Labs:

  • 20.05.2017, 10:05-11:25
 10.

Lecture:

  • Distributed Filesystems

Labs:

Lecture:

  • 20.05.2017, 11:30-12:50

Labs:

  • 20.05.2017, 10:05-11:25
 11.

Lecture:

  • Introduction to cloud computing (lecture)
  • HA+LB Web Project (labs)

Labs:

  • HA+LB Web Project (labs)
Lecture:

  • 20.05.2017, 15:00-16:20

Labs:

  • 21.05.2017, 11:30-12:50
 12.

Lecture:

  • Introduction to Microsoft Active Directory (lecture)

Labs:

  • HA+LB Web Project (labs)
Lecture:

  • 21.05.2017, 16:30-18:00

Labs:

  • 21.05.2017, 08:40-10:00
 13.

Lecture:

  • Virtualization  (lecture)

Labs:

  • HA+LB Web Project (labs)
Lecture:

  • 21.05.2017, 15:00-16:30

Labs:

  • 21.05.2017, 10:05-11:25
14.

Lecture:

  • Containers

Labs:

Lecture:

  • 21.05.2017, 13:20-14:40

Labs:

  • 21.05.2017, 11:30-12:50
 15.

We had only 14 classes scheduled for the semester and at least one classes should be scheduled for lab tests, so 14 is enough 🙂

Nevertheless, for future purposes,

Lecture:

  • No SQL Databases

Labs:

  • HA+LB Web Project – adding NoSQL (nosql databases, dist index querying),

N/A