CS125x: Advanced Distributed Machine Learning with Apache Spark

Brought by: edX

Overview

Building on the core ideas presented in Distributed Machine Learning with Spark, this course covers advanced topics for training and deploying large-scale learning pipelines. You will study state-of-the-art distributed algorithms for collaborative filtering, ensemble methods (e.g., random forests), clustering and topic modeling, with a focus on model parallelism and the crucial tradeoffs between computation and communication.

After completing this course, you will have a thorough understanding of the statistical and algorithmic principles required to develop and deploy distributed machine learning pipelines. You will further have the expertise to write efficient and scalable code in Spark, using MLlib and the spark.ml package in particular.

Taught by

Ameet Talwalkar and Jon Bates

CS125x: Advanced Distributed Machine Learning with Apache Spark
Go to course

CS125x: Advanced Distributed Machine Learning with Apache Spark

Brought by: edX

  • edX
  • Free
  • English
  • Certificate Available
  • Certain days
  • All
  • N/A
8.1.2PHP Version363msRequest Duration2MBMemory UsageGET en/courses/{slug}Route
    • Booting (231ms)
    • Application (131ms)
    • 1 x Booting (63.58%)
      230.73ms
      1 x Application (36.17%)
      131.27ms
      14 templates were rendered
      • public.courses.show (resources/views/public/courses/show.blade.php)3bladefile
        Params
        0
        course
        1
        links
        2
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.courses.partials.details (resources/views/public/courses/partials/details.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.breadcrumbs (resources/views/public/courses/partials/breadcrumbs.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.courses.partials.heading (resources/views/public/courses/partials/heading.blade.php)7bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        classes
      • public.layouts.main (resources/views/public/layouts/main.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.meta (resources/views/public/layouts/partials/meta.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.layouts.partials.navbar (resources/views/public/layouts/partials/navbar.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.links (resources/views/public/auth/profile/partials/links.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.auth.profile.partials.link (resources/views/public/auth/profile/partials/link.blade.php)8bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
        6
        route
        7
        title
      • public.layouts.partials.flash-session (resources/views/public/layouts/partials/flash-session.blade.php)6bladefile
        Params
        0
        __env
        1
        app
        2
        errors
        3
        course
        4
        links
        5
        config
      uri
      GET en/courses/{slug}
      middleware
      web, localize:en
      controller
      App\Http\Controllers\CourseController@show
      as
      en.courses.show
      namespace
      prefix
      /en
      where
      file
      app/Http/Controllers/CourseController.php:17-35
      7 statements were executed7.75ms
      • select * from `courses` where `slug_en` = 'cs125x:-advanced-distributed-machine-learning-with-apache-spark' limit 1
        6.3ms/app/Http/Controllers/CourseController.php:20corspedia
        Metadata
        Bindings
        • 0. cs125x:-advanced-distributed-machine-learning-with-apache-spark
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:20
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • update `courses` set `visitors` = `visitors` + 1, `courses`.`updated_at` = '2025-05-28 23:00:41' where `id` = 2342
        660μs/app/Http/Controllers/CourseController.php:21corspedia
        Metadata
        Bindings
        • 0. 2025-05-28 23:00:41
        • 1. 2342
        Backtrace
        • 17. /app/Http/Controllers/CourseController.php:21
        • 18. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 19. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 20. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `topic_id`, `slug_en`, `slug_ar` from `subjects` where `subjects`.`id` in (4)
        180μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select `id`, `name_en`, `name_ar`, `slug_en`, `slug_ar` from `topics` where `topics`.`id` in (1)
        130μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 25. /app/Http/Controllers/CourseController.php:23
        • 26. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 27. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 28. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 29. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `institutions` where `institutions`.`id` in (65) and `institutions`.`deleted_at` is null
        150μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `providers` where `providers`.`id` in (1) and `providers`.`deleted_at` is null
        140μs/app/Http/Controllers/CourseController.php:23corspedia
        Metadata
        Backtrace
        • 20. /app/Http/Controllers/CourseController.php:23
        • 21. /vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • 22. /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • 23. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • 24. /vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • select * from `html_files` where `html_files`.`id` = 2333 limit 1
        190μs/app/Models/Course.php:84corspedia
        Metadata
        Bindings
        • 0. 2333
        Backtrace
        • 21. /app/Models/Course.php:84
        • 28. view::public.courses.show:29
        • 30. /vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:125
        • 31. /vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • 32. /vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
      App\Models\HtmlFile
      1
      App\Models\Provider
      1
      App\Models\Institution
      1
      App\Models\Topic
      1
      App\Models\Subject
      1
      App\Models\Course
      1
        _token
        0Xu3AP54tSAES0TmnETH39b994RWFxtV8yCEH65b
        locale
        en
        _previous
        array:1 [ "url" => "https://www.corspedia.com/en/courses/cs125x:-advanced-distributed-machine-lear...
        _flash
        array:2 [ "old" => [] "new" => [] ]
        PHPDEBUGBAR_STACK_DATA
        []
        path_info
        /en/courses/cs125x:-advanced-distributed-machine-learning-with-apache-spark
        status_code
        200
        
        status_text
        OK
        format
        html
        content_type
        text/html; charset=UTF-8
        request_query
        []
        
        request_request
        []
        
        request_headers
        0 of 0
        array:24 [ "cf-ipcountry" => array:1 [ 0 => "US" ] "cf-connecting-ip" => array:1 [ 0 => "3.148.227.197" ] "cdn-loop" => array:1 [ 0 => "cloudflare; loops=1" ] "x-forwarded-proto" => array:1 [ 0 => "https" ] "x-forwarded-for" => array:1 [ 0 => "3.148.227.197" ] "sec-fetch-site" => array:1 [ 0 => "none" ] "accept" => array:1 [ 0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" ] "user-agent" => array:1 [ 0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" ] "upgrade-insecure-requests" => array:1 [ 0 => "1" ] "sec-ch-ua-platform" => array:1 [ 0 => ""Windows"" ] "sec-ch-ua-mobile" => array:1 [ 0 => "?0" ] "sec-ch-ua" => array:1 [ 0 => ""Chromium";v="130", "HeadlessChrome";v="130", "Not?A_Brand";v="99"" ] "cache-control" => array:1 [ 0 => "no-cache" ] "pragma" => array:1 [ 0 => "no-cache" ] "sec-fetch-dest" => array:1 [ 0 => "document" ] "cf-ray" => array:1 [ 0 => "94715e9a9eb713f9-ORD" ] "accept-encoding" => array:1 [ 0 => "gzip, br" ] "priority" => array:1 [ 0 => "u=0, i" ] "sec-fetch-user" => array:1 [ 0 => "?1" ] "sec-fetch-mode" => array:1 [ 0 => "navigate" ] "cf-visitor" => array:1 [ 0 => "{"scheme":"https"}" ] "host" => array:1 [ 0 => "www.corspedia.com" ] "content-length" => array:1 [ 0 => "" ] "content-type" => array:1 [ 0 => "" ] ]
        request_server
        0 of 0
        array:50 [ "USER" => "www-data" "HOME" => "/var/www" "HTTP_CF_IPCOUNTRY" => "US" "HTTP_CF_CONNECTING_IP" => "3.148.227.197" "HTTP_CDN_LOOP" => "cloudflare; loops=1" "HTTP_X_FORWARDED_PROTO" => "https" "HTTP_X_FORWARDED_FOR" => "3.148.227.197" "HTTP_SEC_FETCH_SITE" => "none" "HTTP_ACCEPT" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" "HTTP_USER_AGENT" => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "HTTP_UPGRADE_INSECURE_REQUESTS" => "1" "HTTP_SEC_CH_UA_PLATFORM" => ""Windows"" "HTTP_SEC_CH_UA_MOBILE" => "?0" "HTTP_SEC_CH_UA" => ""Chromium";v="130", "HeadlessChrome";v="130", "Not?A_Brand";v="99"" "HTTP_CACHE_CONTROL" => "no-cache" "HTTP_PRAGMA" => "no-cache" "HTTP_SEC_FETCH_DEST" => "document" "HTTP_CF_RAY" => "94715e9a9eb713f9-ORD" "HTTP_ACCEPT_ENCODING" => "gzip, br" "HTTP_PRIORITY" => "u=0, i" "HTTP_SEC_FETCH_USER" => "?1" "HTTP_SEC_FETCH_MODE" => "navigate" "HTTP_CF_VISITOR" => "{"scheme":"https"}" "HTTP_HOST" => "www.corspedia.com" "REDIRECT_STATUS" => "200" "SERVER_NAME" => "corspedia.com" "SERVER_PORT" => "443" "SERVER_ADDR" => "141.95.147.152" "REMOTE_USER" => "" "REMOTE_PORT" => "15552" "REMOTE_ADDR" => "172.70.126.105" "SERVER_SOFTWARE" => "nginx/1.18.0" "GATEWAY_INTERFACE" => "CGI/1.1" "HTTPS" => "on" "REQUEST_SCHEME" => "https" "SERVER_PROTOCOL" => "HTTP/2.0" "DOCUMENT_ROOT" => "/var/www/corspedia/public" "DOCUMENT_URI" => "/index.php" "REQUEST_URI" => "/en/courses/cs125x:-advanced-distributed-machine-learning-with-apache-spark" "SCRIPT_NAME" => "/index.php" "CONTENT_LENGTH" => "" "CONTENT_TYPE" => "" "REQUEST_METHOD" => "GET" "QUERY_STRING" => "" "SCRIPT_FILENAME" => "/var/www/corspedia/public/index.php" "PATH_INFO" => "" "FCGI_ROLE" => "RESPONDER" "PHP_SELF" => "/index.php" "REQUEST_TIME_FLOAT" => 1748473241.0103 "REQUEST_TIME" => 1748473241 ]
        request_cookies
        []
        
        response_headers
        0 of 0
        array:5 [ "content-type" => array:1 [ 0 => "text/html; charset=UTF-8" ] "cache-control" => array:1 [ 0 => "no-cache, private" ] "date" => array:1 [ 0 => "Wed, 28 May 2025 23:00:41 GMT" ] "set-cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6IjV6YjFoRkp3d0x4eVZXYi9FNnJITUE9PSIsInZhbHVlIjoiN3FUNGdzSHVMUzVvVStJc3hYZVJnM0xTUTVPQXNrdzVHSG41RWV4WmY3cm5FQUZOTFlVbnpwNHFSYklqK3pBZDdVRitSZ1IvZVVUbFlzbzhYbEYxb1NiMWFQSVlFaVdRbHFhNlE5bzBKcW1QL0huN2xWaEQ4L3llcUVITXVQUWoiLCJtYWMiOiIyYjdjNzg5MmI5NjY1YTgwYjZiZTQ1NWU2MmU4MjQ2NGIwNTFkZDJkZGFhOTc0N2IwMmYxZjI3YTZiYTI3ZTYxIiwidGFnIjoiIn0%3D; expires=Thu, 29 May 2025 01:00:41 GMT; Max-Age=7200; path=/; samesite=laxXSRF-TOKEN=eyJpdiI6IjV6YjFoRkp3d0x4eVZXYi9FNnJITUE9PSIsInZhbHVlIjoiN3FUNGdzSHVMUzVvVStJc3hYZVJnM0xTUTVPQXNrdzVHSG41RWV4WmY3cm5FQUZOTFlVbnpwNHFSYklqK3pBZDdVRitSZ" 1 => "laravel_session=eyJpdiI6Ik1vRklhTUJXUGpxaXBLT09MYlNjdHc9PSIsInZhbHVlIjoiVmI1STM1L2xiT0t4UWZGYXdFMng3Z1QwK0xHamZTTTB0emdzbW1VT1RDdHo2SG9ZMnhRclJTSGtWa3Y4bjV2cXVFdU5FMTBqQlBMaU5kQUx6U3BvbmxFRy84b21zandRdG5iNkhPeU9Ya2txUmt2bWJKMEhKY3RBUnVIbjl3dUciLCJtYWMiOiIzNjkyMTNhODE2ZTQyM2U2Nzc0Nzg5MDA5YTUxZmE0ZmZkZDI4YmFkZDZmYmNmYTNhMDE2Nzc3MDRmYjRlN2M3IiwidGFnIjoiIn0%3D; expires=Thu, 29 May 2025 01:00:41 GMT; Max-Age=7200; path=/; httponly; samesite=laxlaravel_session=eyJpdiI6Ik1vRklhTUJXUGpxaXBLT09MYlNjdHc9PSIsInZhbHVlIjoiVmI1STM1L2xiT0t4UWZGYXdFMng3Z1QwK0xHamZTTTB0emdzbW1VT1RDdHo2SG9ZMnhRclJTSGtWa3Y4bjV2cXVF" ] "Set-Cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6IjV6YjFoRkp3d0x4eVZXYi9FNnJITUE9PSIsInZhbHVlIjoiN3FUNGdzSHVMUzVvVStJc3hYZVJnM0xTUTVPQXNrdzVHSG41RWV4WmY3cm5FQUZOTFlVbnpwNHFSYklqK3pBZDdVRitSZ1IvZVVUbFlzbzhYbEYxb1NiMWFQSVlFaVdRbHFhNlE5bzBKcW1QL0huN2xWaEQ4L3llcUVITXVQUWoiLCJtYWMiOiIyYjdjNzg5MmI5NjY1YTgwYjZiZTQ1NWU2MmU4MjQ2NGIwNTFkZDJkZGFhOTc0N2IwMmYxZjI3YTZiYTI3ZTYxIiwidGFnIjoiIn0%3D; expires=Thu, 29-May-2025 01:00:41 GMT; path=/XSRF-TOKEN=eyJpdiI6IjV6YjFoRkp3d0x4eVZXYi9FNnJITUE9PSIsInZhbHVlIjoiN3FUNGdzSHVMUzVvVStJc3hYZVJnM0xTUTVPQXNrdzVHSG41RWV4WmY3cm5FQUZOTFlVbnpwNHFSYklqK3pBZDdVRitSZ" 1 => "laravel_session=eyJpdiI6Ik1vRklhTUJXUGpxaXBLT09MYlNjdHc9PSIsInZhbHVlIjoiVmI1STM1L2xiT0t4UWZGYXdFMng3Z1QwK0xHamZTTTB0emdzbW1VT1RDdHo2SG9ZMnhRclJTSGtWa3Y4bjV2cXVFdU5FMTBqQlBMaU5kQUx6U3BvbmxFRy84b21zandRdG5iNkhPeU9Ya2txUmt2bWJKMEhKY3RBUnVIbjl3dUciLCJtYWMiOiIzNjkyMTNhODE2ZTQyM2U2Nzc0Nzg5MDA5YTUxZmE0ZmZkZDI4YmFkZDZmYmNmYTNhMDE2Nzc3MDRmYjRlN2M3IiwidGFnIjoiIn0%3D; expires=Thu, 29-May-2025 01:00:41 GMT; path=/; httponlylaravel_session=eyJpdiI6Ik1vRklhTUJXUGpxaXBLT09MYlNjdHc9PSIsInZhbHVlIjoiVmI1STM1L2xiT0t4UWZGYXdFMng3Z1QwK0xHamZTTTB0emdzbW1VT1RDdHo2SG9ZMnhRclJTSGtWa3Y4bjV2cXVF" ] ]
        session_attributes
        0 of 0
        array:5 [ "_token" => "0Xu3AP54tSAES0TmnETH39b994RWFxtV8yCEH65b" "locale" => "en" "_previous" => array:1 [ "url" => "https://www.corspedia.com/en/courses/cs125x:-advanced-distributed-machine-learning-with-apache-spark" ] "_flash" => array:2 [ "old" => [] "new" => [] ] "PHPDEBUGBAR_STACK_DATA" => [] ]