Package: batchtools 0.9.17

batchtools: Tools for Computation on Batch Systems

As a successor of the packages 'BatchJobs' and 'BatchExperiments', this package provides a parallel implementation of the Map function for high performance computing systems managed by schedulers 'IBM Spectrum LSF' (<https://www.ibm.com/products/hpc-workload-management>), 'OpenLava' (<https://www.openlava.org/>), 'Univa Grid Engine'/'Oracle Grid Engine' (<https://www.univa.com/>), 'Slurm' (<https://slurm.schedmd.com/>), 'TORQUE/PBS' (<https://adaptivecomputing.com/cherry-services/torque-resource-manager/>), or 'Docker Swarm' (<https://docs.docker.com/engine/swarm/>). A multicore and socket mode allow the parallelization on a local machines, and multiple machines can be hooked up via SSH to create a makeshift cluster. Moreover, the package provides an abstraction mechanism to define large-scale computer experiments in a well-organized and reproducible way.

Authors:Michel Lang [cre, aut], Bernd Bischl [aut], Dirk Surmann [ctb]

batchtools_0.9.17.tar.gz
batchtools_0.9.17.zip(r-4.5)batchtools_0.9.17.zip(r-4.4)batchtools_0.9.17.zip(r-4.3)
batchtools_0.9.17.tgz(r-4.4-x86_64)batchtools_0.9.17.tgz(r-4.4-arm64)batchtools_0.9.17.tgz(r-4.3-x86_64)batchtools_0.9.17.tgz(r-4.3-arm64)
batchtools_0.9.17.tar.gz(r-4.5-noble)batchtools_0.9.17.tar.gz(r-4.4-noble)
batchtools_0.9.17.tgz(r-4.4-emscripten)batchtools_0.9.17.tgz(r-4.3-emscripten)
batchtools.pdf |batchtools.html
batchtools/json (API)
NEWS

# Install 'batchtools' in R:
install.packages('batchtools', repos = c('https://mllg.r-universe.dev', 'https://cloud.r-project.org'))

Peer review:

Bug tracker:https://github.com/mllg/batchtools/issues

On CRAN:

batchexperimentsbatchjobsdocker-swarmhigh-performance-computinghpchpc-clusterslsfopenlavaparallel-computingreproducibilitysgeslurmtorque

11.26 score 172 stars 12 packages 748 scripts 2.2k downloads 3 mentions 100 exports 21 dependencies

Last updated 2 years agofrom:1196047ed5. Checks:OK: 2 NOTE: 7. Indexed: yes.

TargetResultDate
Doc / VignettesOKNov 07 2024
R-4.5-win-x86_64NOTENov 07 2024
R-4.5-linux-x86_64NOTENov 07 2024
R-4.4-win-x86_64NOTENov 07 2024
R-4.4-mac-x86_64NOTENov 07 2024
R-4.4-mac-aarch64NOTENov 07 2024
R-4.3-win-x86_64OKNov 07 2024
R-4.3-mac-x86_64NOTENov 07 2024
R-4.3-mac-aarch64NOTENov 07 2024

Exports:addAlgorithmaddExperimentsaddJobTagsaddProblemajoinassertRegistrybatchExportbatchMapbatchMapResultsbatchReducebinpackbtlapplybtmapplycfBrewTemplatecfHandleUnknownSubmitErrorcfKillJobcfReadBrewTemplatechunkclearRegistrydoJobCollectionestimateRuntimesexecJobfindConfFilefindDonefindErrorsfindExperimentsfindExpiredfindJobsfindNotDonefindNotStartedfindNotSubmittedfindOnSystemfindQueuedfindRunningfindStartedfindSubmittedfindTaggedfindTemplateFileflattengetDefaultRegistrygetErrorMessagesgetJobNamesgetJobParsgetJobResourcesgetJobStatusgetJobTablegetJobTagsgetLoggetStatusgetUsedJobTagsgrepLogsijoinkillJobsljoinloadRegistryloadResultlptmakeClusterFunctionsmakeClusterFunctionsDockermakeClusterFunctionsInteractivemakeClusterFunctionsLSFmakeClusterFunctionsMulticoremakeClusterFunctionsOpenLavamakeClusterFunctionsSGEmakeClusterFunctionsSlurmmakeClusterFunctionsSocketmakeClusterFunctionsSSHmakeClusterFunctionsTORQUEmakeExperimentRegistrymakeJobmakeJobCollectionmakeRegistrymakeSubmitJobResultojoinreduceResultsreduceResultsDataTablereduceResultsListremoveAlgorithmsremoveExperimentsremoveJobTagsremoveProblemsremoveRegistryresetJobsrjoinrunHookrunOSCommandsaveRegistrysetDefaultRegistrysetJobNamesshowLogsjoinsubmitJobssummarizeExperimentssweepRegistrysyncRegistrytestJobujoinunwrapwaitForJobsWorker

Dependencies:backportsbase64urlbrewcheckmateclicrayondata.tabledigestfsgluehmslifecyclepkgconfigprettyunitsprogressR6rappdirsrlangstringivctrswithr

batchtools

Rendered frombatchtools.Rmdusingknitr::rmarkdownon Nov 07 2024.

Last update: 2020-12-11
Started: 2017-07-28

Readme and manuals

Help Manual

Help pageTopics
batchtools: Tools for Computation on Batch Systemsbatchtools-package batchtools
Define Algorithms for ExperimentsaddAlgorithm Algorithm removeAlgorithms
Add Experiments to the RegistryaddExperiments
Define Problems for ExperimentsaddProblem Problem removeProblems
assertRegistryassertRegistry
Export Objects to the SlavesbatchExport
Map Operation for Batch SystemsbatchMap
Map Over Results to Create New JobsbatchMapResults
Reduce Operation for Batch SystemsbatchReduce
Synchronous Apply Functionsbtlapply btmapply
Cluster Functions Helper to Write Job Description FilescfBrewTemplate
Cluster Functions Helper to Handle Unknown ErrorscfHandleUnknownSubmitError
Cluster Functions Helper to Kill Batch JobscfKillJob
Cluster Functions Helper to Parse a Brew TemplatecfReadBrewTemplate
Chunk Jobs for Sequential Executionbinpack chunk lpt
Remove All JobsclearRegistry
Execute Jobs of a JobCollectiondoJobCollection
Estimate Remaining RuntimesestimateRuntimes print.RuntimeEstimate
Execute a Single JobsexecJob
Find and Filter JobsfindDone findErrors findExperiments findExpired findJobs findNotDone findNotStarted findNotSubmitted findOnSystem findQueued findRunning findStarted findSubmitted findTagged
Get and Set the Default RegistrygetDefaultRegistry setDefaultRegistry
Retrieve Error MessagesgetErrorMessages
Query Job InformationgetJobPars getJobResources getJobStatus getJobTable getJobTags
Summarize the Computational StatusgetStatus
Grep Log Files for a PatterngrepLogs
Set and Retrieve Job NamesgetJobNames JobNames setJobNames
Inner, Left, Right, Outer, Semi and Anti Join for Data Tablesajoin ijoin JoinTables ljoin ojoin rjoin sjoin ujoin
Kill JobskillJobs
Load a Registry from the File SystemloadRegistry
Load the Result of a Single JobloadResult
ClusterFunctions ConstructorClusterFunctions makeClusterFunctions
ClusterFunctions for DockermakeClusterFunctionsDocker
ClusterFunctions for Sequential Execution in the Running R SessionmakeClusterFunctionsInteractive
ClusterFunctions for LSF SystemsmakeClusterFunctionsLSF
ClusterFunctions for Parallel Multicore ExecutionmakeClusterFunctionsMulticore
ClusterFunctions for OpenLavamakeClusterFunctionsOpenLava
ClusterFunctions for SGE SystemsmakeClusterFunctionsSGE
ClusterFunctions for Slurm SystemsmakeClusterFunctionsSlurm
ClusterFunctions for Parallel Socket ExecutionmakeClusterFunctionsSocket
ClusterFunctions for Remote SSH ExecutionmakeClusterFunctionsSSH
ClusterFunctions for OpenPBS/TORQUE SystemsmakeClusterFunctionsTORQUE
ExperimentRegistry ConstructorExperimentRegistry makeExperimentRegistry
Jobs and ExperimentsExperiment Job makeJob
JobCollection ConstructorJobCollection makeJobCollection
Registry ConstructormakeRegistry Registry
Create a SubmitJobResultmakeSubmitJobResult SubmitJobResult
Reduce ResultsreduceResults
Apply Functions on ResultsreduceResultsDataTable reduceResultsList
Remove ExperimentsremoveExperiments
Remove a Registry from the File SystemremoveRegistry
Reset the Computational State of JobsresetJobs
Trigger Evaluation of Custom FunctionHook Hooks runHook
Run OS Commands on Local or Remote MachinesrunOSCommand
Store the Registy to the File SystemsaveRegistry
Inspect Log FilesgetLog showLog
Submit Jobs to the Batch SystemssubmitJobs
Quick Summary over ExperimentssummarizeExperiments
Check Consistency and Remove Obsolete InformationsweepRegistry
Synchronize the RegistrysyncRegistry
Add or Remove Job TagsaddJobTags getUsedJobTags removeJobTags Tags
Run Jobs InteractivelytestJob
Unwrap Nested Data Framesflatten unwrap
Wait for Termination of JobswaitForJobs
Create a Linux-WorkerWorker