CARVIEW |
Select Language
HTTP/2 200
date: Thu, 09 Oct 2025 19:09:09 GMT
content-type: text/html; charset=utf-8
vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With,Accept-Encoding, Accept, X-Requested-With
etag: W/"ee462a61df33894785ecc8061215be38"
cache-control: max-age=0, private, must-revalidate
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: no-referrer-when-downgrade
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com github.githubassets.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com wss://alive-staging.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com release-assets.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com marketplace-screenshots.githubusercontent.com/ copilotprodattachments.blob.core.windows.net/github-production-copilot-attachments/ github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/
server: github.com
content-encoding: gzip
accept-ranges: bytes
set-cookie: _gh_sess=966ur6%2FpuiiJvxdMjRDdIphZANj0rBo9Tf5zmfd4%2BcyEYxd00x55vQaaLr0OJ0aqEFoqr1uEpuhqz93an45TaoXd393IJpdUk%2F28L5pTSJykpHPLQxqN19d0i2oDuSIyCf0YyxbBy8zlJESrrWCswxSGl0gS2wQ65c8ppyc7wtBP%2F0I58yUGqhFsW2W7x%2FpB5UUaAlrvRebzh%2BXwFubANQO8Ry50FdBwz2y9yJwiX4mLNymAcmpL%2FBEHkyW8jK%2BASmuZ%2FeUJV6Tnot6lJOPQCw%3D%3D--sPx9o%2BJjOuezh53v--D0nlEa7eza5MtahbaiPCzg%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
set-cookie: _octo=GH1.1.1811826752.1760036948; Path=/; Domain=github.com; Expires=Fri, 09 Oct 2026 19:09:08 GMT; Secure; SameSite=Lax
set-cookie: logged_in=no; Path=/; Domain=github.com; Expires=Fri, 09 Oct 2026 19:09:08 GMT; HttpOnly; Secure; SameSite=Lax
x-github-request-id: A176:275E3F:FBC769:129CC58:68E80854
GitHub - documentcloud/cloud-crowd: Parallel Processing for the Rest of Us
Skip to content
Navigation Menu
{{ message }}
-
Notifications
You must be signed in to change notification settings - Fork 85
Parallel Processing for the Rest of Us
License
documentcloud/cloud-crowd
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
= _ _ ( ` )_ ( ) `) (_ (_ . _) _) _ ( ) _ . ( ` ) . ) ( _ )_ (_, _( ,_)_) (_ _(_ ,) _ _ ___ _ _ ___ _ ( ` )_ / __| |___ _ _ __| |/ __|_ _ _____ __ ____| | ( ) `) | (__| / _ \ || / _` | (__| '_/ _ \ V V / _` | (_ (_ . _) _) \___|_\___/\_,_\__,_|\___|_| \___/\_/\_/\__,_| _ ( ) _, _ . ( ` ) . ) ( ( _ )_ (_, _( ,_)_) (_(_ _(_ ,) ~ CloudCrowd ~ * Parallel processing for the rest of us * Write your scripts in Ruby * Works with Amazon EC2 and S3 * split -> process -> merge * As easy as `gem install cloud-crowd` Well-suited for: * Generating or resizing images. * Encoding video. * Running text extraction or OCR on PDFs. * Migrating a large file set or database. * Web scraping. ~ Documentation ~ Wiki: https://github.com/documentcloud/cloud-crowd/wiki Rdoc: https://www.rubydoc.info/github/documentcloud/cloud-crowd ~ Getting started ~ # Install the gem. >> sudo gem install cloud-crowd # Install the CloudCrowd configuration files to a location of your choosing. >> crowd install ~/config/cloud-crowd # Now, you can use the full complement of `crowd` commands from inside of # this configuration directory. To see the available commands: >> crowd --help # Edit the configuration files to your satisfaction, add AWS credentials, # and then load the CloudCrowd schema into your configured database. >> cd ~/config/cloud-crowd >> mate config.yml >> mate database.yml >> [create the database you just configured...] >> crowd load_schema # Write your actions, and install them into the 'actions' subdirectory. # CloudCrowd comes with a few default actions as an example. # To launch the central server (make sure that you include its location # in config.yml): >> crowd server # The configuration folder also includes 'config.ru', which can be used by # any Rack-compliant webserver to run your central server. # Then, to launch a node of workers: >> crowd node # To spin up remote nodes, install the 'cloud-crowd' gem and copy over # your configuration directory. Run `crowd node`, and the remote machines # will register with the central server, becoming available for processing. # At this point you can visit your Operations Center at localhost:9173 to # view all of your nodes, ready for action.
About
Parallel Processing for the Rest of Us
Resources
License
Stars
Watchers
Forks
Packages 0
No packages published
You can’t perform that action at this time.