WrapperApp – BOINC (original) (raw)

  1. The BOINC Wrapper
    1. The job description file
    2. Command-line options
    3. Example
    4. Physical file management
    5. Graphics
    6. GenWrapper: A more general BOINC wrapper

The BOINC Wrapper

An existing application (or sequence of applications) can be run under BOINC using a wrapper program supplied by BOINC. The wrapper runs the applications as subprocesses, and handles all communication with the BOINC client (e.g., to report CPU time and fraction done).

http://boinc.berkeley.edu/images/wrapper.png

The source code of wrapper is in boinc/samples. You can get pre-compiled versions here:

The job description file

The wrapper reads a file with logical name 'job.xml'. This file has the format:

worker [ stdin_file ] [ stdout_file ] [ stderr_file ] [ --foo bar ] [ X ] [ filename ] [ filename ] [ dirname ] [ ] [ VARNAME=VAR_VALUE ] [ ] [ ] [ X ] [ N ] [ other s ] [ foo.zip ... ] [ foo.zip regexp ... ] [ foo ]

The job file describes a sequence of tasks. The descriptor for each task includes:

application

The logical name of the application, or 'worker program'.

stdin_filename, stdout_filename, stderr_filename

The logical names of the files to which stdin, stdout, and stderr are to be connected (if any).

command_line

command-line arguments to be passed to the worker program. This string is macro-substituted as follows:

weight

the contribution of each task to the overall fraction done is proportional to its weight (floating-point, default 1). For example, if your job has tasks A and B, and A uses 100 times more CPU time than B, set A.weight=100 and B.weight=1.

checkpoint_filename

the name of the checkpoint file used by the app, if any. When this is modified, the wrapper assumes that a checkpoint has been completed and notifies the core client.

fraction_done_filename

the name of a file to which the app will periodically write its fraction done (0 to 1). This is used by the wrapper to report overall fraction done.

exec_dir

The directory to start the application in (relative to slot, or use the $PROJECT_DIR macro)

multi_process

Include this if the application creates multiple processes.Note: each parent process must wait for its children to exit.

setenv

Environment variable needed by the applications. You can have more than one entry. Use the VARNAME=VAR_VALUE form, e.g. LD_LIBRARY_PATH=$PROJECT_DIR:$LD_LIBRARY_PATH. You can also use the NTHREADSorNTHREADS or NTHREADSorGPU_DEVICE_NUM macro.

daemon

this task is a 'daemon' process that should run in the background asynchronously while the other tasks are run sequentially. The wrapper will shut down this daemon when the last task has exited.

append_cmdline_args

if set, the wrapper's command-line arguments (specified in the input template) are passed to the worker program, after those in <command_line>.

time_limit

if given, kill the task after the given amount of elapsed (running) time. Note: on Windows, tasks are killed using TerminateProcess(), which doesn't flush stdio buffers; take this into account.

priority

set the process of the task based on N:

Use high priorities only for tasks that use little CPU time.

The job file can specify multiple tasks. This is useful for two purposes:

The wrapper can optionally unzip input files:

unzip_input

before running tasks, the wrapper will unzip the specified input files.

The wrapper can optionally zip output files:

zip_output

after all tasks are completed, the wrapper will zip output files (specified by one or more regular expressions) into a zip file with the given name.

The wrapper can optionally rename output files:

rename_output

for this to work, a result file with <open_name>foo.link</open_name> has to be specified in the result template. After all tasks are completed, a file "foo" created by the application will be moved to become the result file, without using additional disk space or copy time. Note: The link name to be specified in the result template is hardcoded right now but this may change in a future version.

Notes:

Command-line options

The wrapper has the following command-line options:

--device N: macro-substitute N for $GPU_DEVICE_NUM

in worker command lines and environment variables.

--nthreads X: macro-substitute X for $NTHREADS

in worker command lines and environment variables.

--trickle X: send a trickle-up message reporting runtime every X seconds.

Example

Assume you have an executable program for a particular platform (say "worker_windows_intelx86_0.exe" for Win32). The program reads from in and writes to out. (You can use the program in samples/worker/ for this purpose).

We assume that you have already created a projectwith root directory PROJECT/. Now

#! /bin/sh
bin/stage_file input
bin/create_work --appname worker --wu_name worker_nodelete input

to generate a workunit.

Physical file management

You can use the wrapper together with physical file management, where you directly access files in your project directory. For example, you could create a job whose first task unpacks a zip file into the project directory, and whose subsequent tasks access these files.

The support for this is:

Graphics

You can include a graphics app with a wrapper-based application.

GenWrapper: A more general BOINC wrapper

When the functionality of the BOINC Wrapper is not enough, there is a generic solution which uses POSIX-like shell scripting, instead of the XML config file, for describing jobs: You can have complex control flows (loops, branches, etc), but remember "with great power must also come -- great responsibility!"

GenWrapper was developed by a group at SZTAKI in Hungary. The homepage is ​here, and documentation is ​here.