| CARVIEW |
Select Language
HTTP/2 302
server: nginx
date: Mon, 18 Aug 2025 02:22:08 GMT
content-type: text/plain; charset=utf-8
content-length: 0
x-archive-redirect-reason: found capture at 20100211160800
location: https://web.archive.org/web/20100211160800/https://github.com/documentcloud/cloud-crowd
server-timing: captures_list;dur=1.931028, exclusion.robots;dur=0.028012, exclusion.robots.policy;dur=0.012038, esindex;dur=0.015900, cdx.remote;dur=24.220224, LoadShardBlock;dur=145.791424, PetaboxLoader3.datanode;dur=42.881899, PetaboxLoader3.resolve;dur=35.358436
x-app-server: wwwb-app219
x-ts: 302
x-tr: 237
server-timing: TR;dur=0,Tw;dur=0,Tc;dur=0
set-cookie: wb-p-SERVER=wwwb-app219; path=/
x-location: All
x-rl: 0
x-na: 0
x-page-cache: MISS
server-timing: MISS
x-nid: DigitalOcean
referrer-policy: no-referrer-when-downgrade
permissions-policy: interest-cohort=()
HTTP/2 200
server: nginx
date: Mon, 18 Aug 2025 02:22:08 GMT
content-type: text/html; charset=utf-8
x-archive-orig-server: nginx/0.7.61
x-archive-orig-date: Thu, 11 Feb 2010 16:08:00 GMT
x-archive-orig-connection: close
x-archive-orig-status: 200 OK
x-archive-orig-etag: "34ad3e774b92200128953bba7df8f1ec"
x-archive-orig-x-runtime: 171ms
x-archive-orig-content-length: 36172
x-archive-orig-cache-control: private, max-age=0, must-revalidate
x-archive-guessed-content-type: text/html
x-archive-guessed-charset: utf-8
memento-datetime: Thu, 11 Feb 2010 16:08:00 GMT
link: ; rel="original", ; rel="timemap"; type="application/link-format", ; rel="timegate", ; rel="first memento"; datetime="Tue, 22 Sep 2009 18:43:46 GMT", ; rel="prev memento"; datetime="Thu, 07 Jan 2010 08:19:46 GMT", ; rel="memento"; datetime="Thu, 11 Feb 2010 16:08:00 GMT", ; rel="next memento"; datetime="Thu, 18 Mar 2010 02:05:54 GMT", ; rel="last memento"; datetime="Fri, 11 Jul 2025 16:32:18 GMT"
content-security-policy: default-src 'self' 'unsafe-eval' 'unsafe-inline' data: blob: archive.org web.archive.org web-static.archive.org wayback-api.archive.org athena.archive.org analytics.archive.org pragma.archivelab.org wwwb-events.archive.org
x-archive-src: 51_14_20100211133235_crawl102-c/51_14_20100211160744_crawl101.arc.gz
server-timing: captures_list;dur=0.778973, exclusion.robots;dur=0.024185, exclusion.robots.policy;dur=0.010788, esindex;dur=0.020187, cdx.remote;dur=5.533118, LoadShardBlock;dur=173.639830, PetaboxLoader3.datanode;dur=203.267643, PetaboxLoader3.resolve;dur=96.523572, load_resource;dur=137.826851
x-app-server: wwwb-app219
x-ts: 200
x-tr: 387
server-timing: TR;dur=0,Tw;dur=0,Tc;dur=0
x-location: All
x-rl: 0
x-na: 0
x-page-cache: MISS
server-timing: MISS
x-nid: DigitalOcean
referrer-policy: no-referrer-when-downgrade
permissions-policy: interest-cohort=()
content-encoding: gzip
documentcloud's cloud-crowd at master - GitHub
This service is courtesy of Pledgie.
documentcloud / cloud-crowd
- Source
- Commits
- Network (15)
- Issues (2)
- Downloads (14)
- Wiki (10)
- Graphs
-
Branch:
master
click here to add a description
click here to add a homepage
Sending Request…
Enable Donations
Pledgie Donations
Once activated, we'll place the following badge in your repository's detail box:
Parallel Processing for the Rest of Us — Read more
README
=
_ _
( ` )_
( ) `)
(_ (_ . _) _)
_
( )
_ . ( ` ) . )
( _ )_ (_, _( ,_)_)
(_ _(_ ,)
_ _ ___ _ _ ___ _
( ` )_ / __| |___ _ _ __| |/ __|_ _ _____ __ ____| |
( ) `) | (__| / _ \ || / _` | (__| '_/ _ \ V V / _` |
(_ (_ . _) _) \___|_\___/\_,_\__,_|\___|_| \___/\_/\_/\__,_|
_
( )
_, _ . ( ` ) . )
( ( _ )_ (_, _( ,_)_)
(_(_ _(_ ,)
~ CloudCrowd ~
* Parallel processing for the rest of us
* Write your scripts in Ruby
* Works with Amazon EC2 and S3
* split -> process -> merge
* As easy as `gem install cloud-crowd`
Well-suited for:
* Generating or resizing images.
* Encoding video.
* Running text extraction or OCR on PDFs.
* Migrating a large file set or database.
* Web scraping.
~ Documentation ~
Wiki: https://wiki.github.com/documentcloud/cloud-crowd
Rdoc: https://rdoc.info/projects/documentcloud/cloud-crowd
~ Getting started ~
# Install the gem.
>> sudo gem install cloud-crowd
# Install the CloudCrowd configuration files to a location of your choosing.
>> crowd install ~/config/cloud-crowd
# Now, you can use the full complement of `crowd` commands from inside of
# this configuration directory. To see the available commands:
>> crowd --help
# Edit the configuration files to your satisfaction, add AWS credentials,
# and then load the CloudCrowd schema into your configured database.
>> cd ~/config/cloud-crowd
>> mate config.yml
>> mate database.yml
>> [create the database you just configured...]
>> crowd load_schema
# Write your actions, and install them into the 'actions' subdirectory.
# CloudCrowd comes with a few default actions as an example.
# To launch the central server (make sure that you include its location
# in config.yml):
>> crowd server
# The configuration folder also includes 'config.ru', which can be used by
# any Rack-compliant webserver to run your central server.
# Then, to launch a node of workers:
>> crowd node
# To spin up remote nodes, install the 'cloud-crowd' gem and copy over
# your configuration directory. Run `crowd node`, and the remote machines
# will register with the central server, becoming available for processing.
# At this point you can visit your Operations Center at localhost:9173 to
# view all of your nodes, ready for action.