| CARVIEW |
Select Language
HTTP/2 302
server: nginx
date: Sat, 16 Aug 2025 03:48:48 GMT
content-type: text/plain; charset=utf-8
content-length: 0
x-archive-redirect-reason: found capture at 20100420141610
location: https://web.archive.org/web/20100420141610/https://github.com/documentcloud/cloud-crowd
server-timing: captures_list;dur=0.819476, exclusion.robots;dur=0.025984, exclusion.robots.policy;dur=0.011241, esindex;dur=0.014361, cdx.remote;dur=1072.884656, LoadShardBlock;dur=394.155275, PetaboxLoader3.datanode;dur=48.968252, PetaboxLoader3.resolve;dur=144.951440
x-app-server: wwwb-app210
x-ts: 302
x-tr: 1509
server-timing: TR;dur=0,Tw;dur=0,Tc;dur=0
set-cookie: wb-p-SERVER=wwwb-app210; path=/
x-location: All
x-rl: 0
x-na: 0
x-page-cache: MISS
server-timing: MISS
x-nid: DigitalOcean
referrer-policy: no-referrer-when-downgrade
permissions-policy: interest-cohort=()
HTTP/2 200
server: nginx
date: Sat, 16 Aug 2025 03:48:49 GMT
content-type: text/html; charset=utf-8
x-archive-orig-server: nginx/0.7.61
x-archive-orig-date: Tue, 20 Apr 2010 14:16:10 GMT
x-archive-orig-connection: close
x-archive-orig-status: 200 OK
x-archive-orig-etag: "b8c8a2ffadc9fb499a89297da7f575f0"
x-archive-orig-x-runtime: 123ms
x-archive-orig-content-length: 34185
x-archive-orig-cache-control: private, max-age=0, must-revalidate
x-archive-guessed-content-type: text/html
x-archive-guessed-charset: utf-8
memento-datetime: Tue, 20 Apr 2010 14:16:10 GMT
link: ; rel="original", ; rel="timemap"; type="application/link-format", ; rel="timegate", ; rel="first memento"; datetime="Tue, 22 Sep 2009 18:43:46 GMT", ; rel="prev memento"; datetime="Thu, 18 Mar 2010 02:05:54 GMT", ; rel="memento"; datetime="Tue, 20 Apr 2010 14:16:10 GMT", ; rel="next memento"; datetime="Wed, 21 Apr 2010 19:45:51 GMT", ; rel="last memento"; datetime="Fri, 11 Jul 2025 16:32:18 GMT"
content-security-policy: default-src 'self' 'unsafe-eval' 'unsafe-inline' data: blob: archive.org web.archive.org web-static.archive.org wayback-api.archive.org athena.archive.org analytics.archive.org pragma.archivelab.org wwwb-events.archive.org
x-archive-src: 51_15_20100420123905_crawl102_IndexOnly-c/51_15_20100420141431_crawl101.arc.gz
server-timing: captures_list;dur=0.682564, exclusion.robots;dur=0.022057, exclusion.robots.policy;dur=0.010209, esindex;dur=0.013528, cdx.remote;dur=19.757541, LoadShardBlock;dur=165.593229, PetaboxLoader3.datanode;dur=125.508657, PetaboxLoader3.resolve;dur=212.112017, load_resource;dur=200.124439
x-app-server: wwwb-app210
x-ts: 200
x-tr: 455
server-timing: TR;dur=0,Tw;dur=0,Tc;dur=0
x-location: All
x-rl: 0
x-na: 0
x-page-cache: MISS
server-timing: MISS
x-nid: DigitalOcean
referrer-policy: no-referrer-when-downgrade
permissions-policy: interest-cohort=()
content-encoding: gzip
documentcloud's cloud-crowd at master - GitHub
documentcloud / cloud-crowd
- Source
- Commits
- Network (20)
- Issues (0)
- Downloads (15)
- Wiki (10)
- Graphs
-
Branch:
master
click here to add a description
click here to add a homepage
README
=
_ _
( ` )_
( ) `)
(_ (_ . _) _)
_
( )
_ . ( ` ) . )
( _ )_ (_, _( ,_)_)
(_ _(_ ,)
_ _ ___ _ _ ___ _
( ` )_ / __| |___ _ _ __| |/ __|_ _ _____ __ ____| |
( ) `) | (__| / _ \ || / _` | (__| '_/ _ \ V V / _` |
(_ (_ . _) _) \___|_\___/\_,_\__,_|\___|_| \___/\_/\_/\__,_|
_
( )
_, _ . ( ` ) . )
( ( _ )_ (_, _( ,_)_)
(_(_ _(_ ,)
~ CloudCrowd ~
* Parallel processing for the rest of us
* Write your scripts in Ruby
* Works with Amazon EC2 and S3
* split -> process -> merge
* As easy as `gem install cloud-crowd`
Well-suited for:
* Generating or resizing images.
* Encoding video.
* Running text extraction or OCR on PDFs.
* Migrating a large file set or database.
* Web scraping.
~ Documentation ~
Wiki: https://wiki.github.com/documentcloud/cloud-crowd
Rdoc: https://rdoc.info/projects/documentcloud/cloud-crowd
~ Getting started ~
# Install the gem.
>> sudo gem install cloud-crowd
# Install the CloudCrowd configuration files to a location of your choosing.
>> crowd install ~/config/cloud-crowd
# Now, you can use the full complement of `crowd` commands from inside of
# this configuration directory. To see the available commands:
>> crowd --help
# Edit the configuration files to your satisfaction, add AWS credentials,
# and then load the CloudCrowd schema into your configured database.
>> cd ~/config/cloud-crowd
>> mate config.yml
>> mate database.yml
>> [create the database you just configured...]
>> crowd load_schema
# Write your actions, and install them into the 'actions' subdirectory.
# CloudCrowd comes with a few default actions as an example.
# To launch the central server (make sure that you include its location
# in config.yml):
>> crowd server
# The configuration folder also includes 'config.ru', which can be used by
# any Rack-compliant webserver to run your central server.
# Then, to launch a node of workers:
>> crowd node
# To spin up remote nodes, install the 'cloud-crowd' gem and copy over
# your configuration directory. Run `crowd node`, and the remote machines
# will register with the central server, becoming available for processing.
# At this point you can visit your Operations Center at localhost:9173 to
# view all of your nodes, ready for action.