Derived class within the interface class hierarchy for supporting interfaces to simulation codes. More...
Public Member Functions | |
ApplicationInterface (const ProblemDescDB &problem_db) | |
constructor | |
~ApplicationInterface () | |
destructor | |
Protected Member Functions | |
void | init_communicators (const IntArray &message_lengths, const int &max_iterator_concurrency) |
allocate communicator partitions for concurrent evaluations within an iterator and concurrent multiprocessor analyses within an evaluation. | |
void | set_communicators (const IntArray &message_lengths) |
set the local parallel partition data for an interface (the partitions are already allocated in ParallelLibrary). | |
void | free_communicators () |
deallocate communicator partitions for concurrent evaluations within an iterator and concurrent multiprocessor analyses within an evaluation. | |
void | init_serial () |
int | asynch_local_evaluation_concurrency () const |
return asynchLocalEvalConcurrency | |
String | interface_synchronization () const |
return interfaceSynchronization | |
void | map (const Variables &vars, const ActiveSet &set, Response &response, const bool asynch_flag=false) |
Provides a "mapping" of variables to responses using a simulation. Protected due to Interface letter-envelope idiom. | |
void | manage_failure (const Variables &vars, const ActiveSet &set, Response &response, int failed_eval_id) |
manages a simulation failure using abort/retry/recover/continuation | |
const IntResponseMap & | synch () |
executes a blocking schedule for asynchronous evaluations in the beforeSynchCorePRPQueue and returns all jobs | |
const IntResponseMap & | synch_nowait () |
executes a nonblocking schedule for asynchronous evaluations in the beforeSynchCorePRPQueue and returns a partial set of completed jobs | |
void | serve_evaluations () |
run on evaluation servers to serve the iterator master | |
void | stop_evaluation_servers () |
used by the iterator master to terminate evaluation servers | |
virtual void | derived_map (const Variables &vars, const ActiveSet &set, Response &response, int fn_eval_id) |
Called by map() and other functions to execute the simulation in synchronous mode. The portion of performing an evaluation that is specific to a derived class. | |
virtual void | derived_map_asynch (const ParamResponsePair &pair) |
Called by map() and other functions to execute the simulation in asynchronous mode. The portion of performing an asynchronous evaluation that is specific to a derived class. | |
virtual void | derived_synch (PRPQueue &prp_queue) |
For asynchronous function evaluations, this method is used to detect completion of jobs and process their results. It provides the processing code that is specific to derived classes. This version waits for at least one completion. | |
virtual void | derived_synch_nowait (PRPQueue &prp_queue) |
For asynchronous function evaluations, this method is used to detect completion of jobs and process their results. It provides the processing code that is specific to derived classes. This version is nonblocking and will return without any completions if none are immediately available. | |
void | self_schedule_analyses () |
blocking self-schedule of all analyses within a function evaluation using message passing | |
void | serve_analyses_synch () |
serve the master analysis scheduler and manage one synchronous analysis job at a time | |
virtual int | derived_synchronous_local_analysis (const int &analysis_id) |
Execute a particular analysis (identified by analysis_id) synchronously on the local processor. Used for the derived class specifics within ApplicationInterface::serve_analyses_synch(). | |
Protected Attributes | |
ParallelLibrary & | parallelLib |
reference to the ParallelLibrary object used to manage MPI partitions for the concurrent evaluations and concurrent analyses parallelism levels | |
bool | suppressOutput |
flag for suppressing output on slave processors | |
int | evalCommSize |
size of evalComm | |
int | evalCommRank |
processor rank within evalComm | |
int | evalServerId |
evaluation server identifier | |
bool | eaDedMasterFlag |
flag for dedicated master partitioning at ea level | |
int | analysisCommSize |
size of analysisComm | |
int | analysisCommRank |
processor rank within analysisComm | |
int | analysisServerId |
analysis server identifier | |
int | numAnalysisServers |
number of analysis servers | |
bool | multiProcAnalysisFlag |
flag for multiprocessor analysis partitions | |
bool | asynchLocalAnalysisFlag |
flag for asynchronous local parallelism of analyses | |
int | asynchLocalAnalysisConcurrency |
limits the number of concurrent analyses in asynchronous local scheduling and specifies hybrid concurrency when message passing | |
int | numAnalysisDrivers |
the number of analysis drivers used for each function evaluation (from the analysis_drivers interface specification) | |
IntSet | completionSet |
the set of completed fn_eval_id's populated by derived_synch() and derived_synch_nowait() | |
Private Member Functions | |
bool | duplication_detect (const Variables &vars, Response &response, const bool asynch_flag) |
checks data_pairs and beforeSynchCorePRPQueue to see if the current evaluation request has already been performed or queued | |
void | self_schedule_evaluations () |
blocking self-schedule of all evaluations in beforeSynchCorePRPQueue using message passing; executes on iteratorComm master | |
void | static_schedule_evaluations () |
blocking static schedule of all evaluations in beforeSynchCorePRPQueue using message passing; executes on iteratorComm master | |
void | asynchronous_local_evaluations (PRPQueue &prp_queue) |
perform all jobs in prp_queue using asynchronous approaches on the local processor | |
void | asynchronous_local_evaluations_static (PRPQueue &prp_queue) |
perform all the jobs in prp_queue using asynchronous approaches on the local processor, but schedule statically such that eval_id is always replaced with an equivalent one, modulo asynchLocalEvalConcurrency | |
void | synchronous_local_evaluations (PRPQueue &prp_queue) |
perform all jobs in prp_queue using synchronous approaches on the local processor | |
void | asynchronous_local_evaluations_nowait (PRPQueue &prp_queue) |
launch new jobs in prp_queue asynchronously (if capacity is available), perform nonblocking query of all running jobs, and process any completed jobs (handles both local self- and local static-scheduling cases) | |
void | serve_evaluations_synch () |
serve the evaluation message passing schedulers and perform one synchronous evaluation at a time | |
void | serve_evaluations_asynch () |
serve the evaluation message passing schedulers and manage multiple asynchronous evaluations | |
void | serve_evaluations_peer () |
serve the evaluation message passing schedulers and perform one synchronous evaluation at a time as part of the 1st peer | |
void | set_evaluation_communicators (const IntArray &message_lengths) |
convenience function for updating the local evaluation partition data following ParallelLibrary::init_evaluation_communicators(). | |
void | set_analysis_communicators () |
convenience function for updating the local analysis partition data following ParallelLibrary::init_analysis_communicators(). | |
void | check_configuration (const int &max_iterator_concurrency) |
perform some error checks on the parallel configuration | |
const ParamResponsePair & | get_source_pair (const Variables &target_vars) |
convenience function for the continuation approach in manage_failure() for finding the nearest successful "source" evaluation to the failed "target" | |
void | continuation (const Variables &target_vars, const ActiveSet &set, Response &response, const ParamResponsePair &source_pair, int failed_eval_id) |
performs a 0th order continuation method to step from a successful "source" evaluation to the failed "target". Invoked by manage_failure() for failAction == "continuation". | |
void | common_input_filtering (const Variables &vars) |
common input filtering operations, e.g. mesh movement | |
void | common_output_filtering (Response &response) |
common output filtering operations, e.g. data filtering | |
Private Attributes | |
int | worldSize |
size of MPI_COMM_WORLD | |
int | worldRank |
processor rank within MPI_COMM_WORLD | |
int | iteratorCommSize |
size of iteratorComm | |
int | iteratorCommRank |
processor rank within iteratorComm | |
bool | ieMessagePass |
flag for message passing at ie scheduling level | |
int | numEvalServers |
number of evaluation servers | |
bool | eaMessagePass |
flag for message passing at ea scheduling level | |
int | procsPerAnalysis |
processors per analysis servers | |
int | lenVarsMessage |
length of a MPIPackBuffer containing a Variables object; computed in Model::init_communicators() | |
int | lenVarsActSetMessage |
length of a MPIPackBuffer containing a Variables object and an ActiveSet object; computed in Model::init_communicators() | |
int | lenResponseMessage |
length of a MPIPackBuffer containing a Response object; computed in Model::init_communicators() | |
int | lenPRPairMessage |
length of a MPIPackBuffer containing a ParamResponsePair object; computed in Model::init_communicators() | |
String | evalScheduling |
user specification of evaluation scheduling algorithm (self, static, or no spec). Used for manual overrides of the auto-configure logic in ParallelLibrary::resolve_inputs(). | |
String | analysisScheduling |
user specification of analysis scheduling algorithm (self, static, or no spec). Used for manual overrides of the auto-configure logic in ParallelLibrary::resolve_inputs(). | |
int | asynchLocalEvalConcurrency |
limits the number of concurrent evaluations in asynchronous local scheduling and specifies hybrid concurrency when message passing | |
bool | asynchLocalEvalStatic |
whether the asynchronous local evaluations are to be performed with a static schedule (default false) | |
IntArray | localServerJobMap |
array with one entry per local "server" indicating the job (fn_eval_id) currently running on the server (used for asynchronour local static schedules) | |
String | interfaceSynchronization |
interface synchronization specification: synchronous (default) or asynchronous | |
bool | headerFlag |
used by synch_nowait to manage output frequency (since this function may be called many times prior to any completions) | |
bool | asvControlFlag |
used to manage a user request to deactivate the active set vector control. true = modify the ASV each evaluation as appropriate (default); false = ASV values are static so that the user need not check them on each evaluation. | |
bool | evalCacheFlag |
used to manage a user request to deactivate the function evaluation cache (i.e., queries and insertions using the data_pairs cache). | |
bool | restartFileFlag |
used to manage a user request to deactivate the restart file (i.e., insertions into write_restart). | |
ShortArray | defaultASV |
the static ASV values used when the user has selected asvControl = off | |
String | failAction |
mitigation action for captured simulation failures: abort, retry, recover, or continuation | |
int | failRetryLimit |
limit on the number of retries for the retry failAction | |
RealVector | failRecoveryFnVals |
the dummy function values used for the recover failAction | |
IntResponseMap | historyDuplicateMap |
used to bookkeep asynchronous evaluations which duplicate data_pairs evaluations. Map key is fnEvalId, map value is corresponding response. | |
std::map< int, std::pair < PRPQueueHIter, Response > > | beforeSynchDuplicateMap |
used to bookkeep fnEvalId, beforeSynchCorePRPQueue iterator, and response of asynchronous evaluations which duplicate queued beforeSynchCorePRPQueue evaluations | |
PRPQueue | beforeSynchCorePRPQueue |
used to bookkeep vars/set/response of nonduplicate asynchronous core evaluations. This is the queue of jobs populated by asynchronous map() that is later scheduled in synch() or synch_nowait(). | |
PRPQueue | beforeSynchAlgPRPQueue |
used to bookkeep vars/set/response of asynchronous algebraic evaluations. This is the queue of algebraic jobs populated by asynchronous map() that is later evaluated in synch() or synch_nowait(). | |
IntSet | runningSet |
used by asynchronous_local_nowait to bookkeep which jobs are running |
Derived class within the interface class hierarchy for supporting interfaces to simulation codes.
ApplicationInterface provides an interface class for performing parameter to response mappings using simulation code(s). It provides common functionality for a number of derived classes and contains the majority of all of the scheduling algorithms in DAKOTA. The derived classes provide the specifics for managing code invocations using system calls, forks, direct procedure calls, or distributed resource facilities.
void init_serial | ( | ) | [inline, protected, virtual] |
DataInterface.C defaults of 0 servers are needed to distinguish an explicit user request for 1 server (serialization of a parallelism level) from no user request (use parallel auto-config). This default causes problems when init_communicators() is not called for an interface object (e.g., static scheduling fails in DirectApplicInterface::derived_map() for NestedModel::optionalInterface). This is the reason for this function: to reset certain defaults for interface objects that are used serially.
Reimplemented from Interface.
References ApplicationInterface::numAnalysisServers, and ApplicationInterface::numEvalServers.
void map | ( | const Variables & | vars, |
const ActiveSet & | set, | ||
Response & | response, | ||
const bool | asynch_flag = false |
||
) | [protected, virtual] |
Provides a "mapping" of variables to responses using a simulation. Protected due to Interface letter-envelope idiom.
The function evaluator for application interfaces. Called from derived_compute_response() and derived_asynch_compute_response() in derived Model classes. If asynch_flag is not set, perform a blocking evaluation (using derived_map()). If asynch_flag is set, add the job to the beforeSynchCorePRPQueue queue for execution by one of the scheduler routines in synch() or synch_nowait(). Duplicate function evaluations are detected with duplication_detect().
Reimplemented from Interface.
References Response::active_set(), Interface::algebraic_mappings(), Interface::algebraicMappings, Interface::asv_mapping(), ApplicationInterface::asvControlFlag, ParallelLibrary::bcast_e(), ApplicationInterface::beforeSynchAlgPRPQueue, ApplicationInterface::beforeSynchCorePRPQueue, Response::copy(), Interface::coreMappings, Dakota::data_pairs, ApplicationInterface::defaultASV, ApplicationInterface::derived_map(), ApplicationInterface::duplication_detect(), ApplicationInterface::evalCacheFlag, Interface::fineGrainEvalCounters, Interface::fnEvalId, Interface::fnGradCounter, Interface::fnHessCounter, Interface::fnLabels, Interface::fnValCounter, Response::function_labels(), Interface::idInterface, Interface::init_algebraic_mappings(), ApplicationInterface::lenVarsActSetMessage, ApplicationInterface::manage_failure(), Interface::multiProcEvalFlag, Interface::newFnEvalId, Interface::newFnGradCounter, Interface::newFnHessCounter, Interface::newFnValCounter, Interface::outputLevel, ApplicationInterface::parallelLib, ActiveSet::request_vector(), Interface::response_mapping(), ApplicationInterface::restartFileFlag, and Dakota::write_restart.
const IntResponseMap & synch | ( | ) | [protected, virtual] |
executes a blocking schedule for asynchronous evaluations in the beforeSynchCorePRPQueue and returns all jobs
This function provides blocking synchronization for all cases of asynchronous evaluations, including the local asynchronous case (background system call, nonblocking fork, & multithreads), the message passing case, and the hybrid case. Called from derived_synchronize() in derived Model classes.
Reimplemented from Interface.
References Interface::algebraic_mappings(), Interface::algebraicMappings, ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::asynchLocalEvalStatic, ApplicationInterface::asynchronous_local_evaluations(), ApplicationInterface::asynchronous_local_evaluations_static(), ApplicationInterface::beforeSynchAlgPRPQueue, ApplicationInterface::beforeSynchCorePRPQueue, ApplicationInterface::beforeSynchDuplicateMap, Interface::coreMappings, ApplicationInterface::historyDuplicateMap, Interface::ieDedMasterFlag, ApplicationInterface::ieMessagePass, Interface::outputLevel, Interface::rawResponseMap, Interface::response_mapping(), ApplicationInterface::self_schedule_evaluations(), and ApplicationInterface::static_schedule_evaluations().
const IntResponseMap & synch_nowait | ( | ) | [protected, virtual] |
executes a nonblocking schedule for asynchronous evaluations in the beforeSynchCorePRPQueue and returns a partial set of completed jobs
This function will eventually provide nonblocking synchronization for all cases of asynchronous evaluations, however it currently supports only the local asynchronous case since nonblocking message passing schedulers have not yet been implemented. Called from derived_synchronize_nowait() in derived Model classes.
Reimplemented from Interface.
References Dakota::abort_handler(), Interface::algebraic_mappings(), Interface::algebraicMappings, ApplicationInterface::asynchronous_local_evaluations_nowait(), ApplicationInterface::beforeSynchAlgPRPQueue, ApplicationInterface::beforeSynchCorePRPQueue, ApplicationInterface::beforeSynchDuplicateMap, Interface::coreMappings, ParamResponsePair::eval_id(), ApplicationInterface::headerFlag, ApplicationInterface::historyDuplicateMap, ApplicationInterface::ieMessagePass, Dakota::lookup_by_eval_id(), Interface::outputLevel, ParamResponsePair::prp_response(), Interface::rawResponseMap, Interface::response_mapping(), and Response::update().
void serve_evaluations | ( | ) | [protected, virtual] |
run on evaluation servers to serve the iterator master
Invoked by the serve() function in derived Model classes. Passes control to serve_evaluations_asynch(), serve_evaluations_peer(), or serve_evaluations_synch() according to specified concurrency and self/static scheduler configuration.
Reimplemented from Interface.
References ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::evalServerId, Interface::ieDedMasterFlag, ApplicationInterface::serve_evaluations_asynch(), ApplicationInterface::serve_evaluations_peer(), and ApplicationInterface::serve_evaluations_synch().
void stop_evaluation_servers | ( | ) | [protected, virtual] |
used by the iterator master to terminate evaluation servers
This code is executed on the iteratorComm rank 0 processor when iteration on a particular model is complete. It sends a termination signal (tag = 0 instead of a valid fn_eval_id) to each of the slave analysis servers. NOTE: This function is called from the Strategy layer even when in serial mode. Therefore, use iteratorCommSize to provide appropriate fall through behavior.
Reimplemented from Interface.
References ParallelLibrary::bcast_e(), ParallelLibrary::free(), Interface::ieDedMasterFlag, ParallelLibrary::isend_ie(), ApplicationInterface::iteratorCommSize, Interface::multiProcEvalFlag, ApplicationInterface::numEvalServers, and ApplicationInterface::parallelLib.
void self_schedule_analyses | ( | ) | [protected] |
blocking self-schedule of all analyses within a function evaluation using message passing
This code is called from derived classes to provide the master portion of a master-slave algorithm for the dynamic self-scheduling of analyses among slave servers. It is patterned after self_schedule_evaluations(). It performs no analyses locally and matches either serve_analyses_synch() or serve_analyses_asynch() on the slave servers, depending on the value of asynchLocalAnalysisConcurrency. Self-scheduling approach assigns jobs in 2 passes. The 1st pass gives each server the same number of jobs (equal to asynchLocalAnalysisConcurrency). The 2nd pass assigns the remaining jobs to slave servers as previous jobs are completed. Single- and multilevel parallel use intra- and inter-communicators, respectively, for send/receive. Specific syntax is encapsulated within ParallelLibrary.
References ApplicationInterface::asynchLocalAnalysisConcurrency, ParallelLibrary::free(), ParallelLibrary::irecv_ea(), ParallelLibrary::isend_ea(), ApplicationInterface::numAnalysisDrivers, ApplicationInterface::numAnalysisServers, ApplicationInterface::parallelLib, ParallelLibrary::waitall(), and ParallelLibrary::waitsome().
Referenced by DirectApplicInterface::derived_map(), ForkApplicInterface::fork_application(), and SysCallApplicInterface::spawn_application().
void serve_analyses_synch | ( | ) | [protected] |
serve the master analysis scheduler and manage one synchronous analysis job at a time
This code is called from derived classes to run synchronous analyses on slave processors. The slaves receive requests (blocking receive), do local derived_map_ac's, and return codes. This is done continuously until a termination signal is received from the master. It is patterned after serve_evaluations_synch().
References ApplicationInterface::analysisCommRank, ParallelLibrary::bcast_a(), ApplicationInterface::derived_synchronous_local_analysis(), ParallelLibrary::isend_ea(), ApplicationInterface::multiProcAnalysisFlag, ApplicationInterface::parallelLib, ParallelLibrary::recv_ea(), and ParallelLibrary::wait().
Referenced by DirectApplicInterface::derived_map(), ForkApplicInterface::fork_application(), and SysCallApplicInterface::spawn_application().
bool duplication_detect | ( | const Variables & | vars, |
Response & | response, | ||
const bool | asynch_flag | ||
) | [private] |
checks data_pairs and beforeSynchCorePRPQueue to see if the current evaluation request has already been performed or queued
Called from map() to check incoming evaluation request for duplication with content of data_pairs and beforeSynchCorePRPQueue. If duplication is detected, return true, else return false. Manage bookkeeping with historyDuplicateMap and beforeSynchDuplicateMap. Note that the list searches can get very expensive if a long list is searched on every new function evaluation (either from a large number of previous jobs, a large number of pending jobs, or both). For this reason, a user request for deactivation of the evaluation cache results in a complete bypass of duplication_detect(), even though a beforeSynchCorePRPQueue search would still be meaningful. Since the intent of this request is to streamline operations, both list searches are bypassed.
References Response::active_set(), ApplicationInterface::beforeSynchCorePRPQueue, ApplicationInterface::beforeSynchDuplicateMap, Response::copy(), Dakota::data_pairs, Interface::fnEvalId, Dakota::hashedQueueEnd(), ApplicationInterface::historyDuplicateMap, Interface::idInterface, Dakota::lookup_by_val(), and Response::update().
Referenced by ApplicationInterface::map().
void self_schedule_evaluations | ( | ) | [private] |
blocking self-schedule of all evaluations in beforeSynchCorePRPQueue using message passing; executes on iteratorComm master
This code is called from synch() to provide the master portion of a master-slave algorithm for the dynamic self-scheduling of evaluations among slave servers. It performs no evaluations locally and matches either serve_evaluations_synch() or serve_evaluations_asynch() on the slave servers, depending on the value of asynchLocalEvalConcurrency. Self-scheduling approach assigns jobs in 2 passes. The 1st pass gives each server the same number of jobs (equal to asynchLocalEvalConcurrency). The 2nd pass assigns the remaining jobs to slave servers as previous jobs are completed. Single- and multilevel parallel use intra- and inter-communicators, respectively, for send/receive. Specific syntax is encapsulated within ParallelLibrary.
References ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::beforeSynchCorePRPQueue, Dakota::data_pairs, ApplicationInterface::evalCacheFlag, ParallelLibrary::free(), ParallelLibrary::irecv_ie(), ParallelLibrary::isend_ie(), ApplicationInterface::lenResponseMessage, Dakota::lookup_by_eval_id(), ApplicationInterface::numEvalServers, Interface::outputLevel, ApplicationInterface::parallelLib, Interface::rawResponseMap, MPIUnpackBuffer::reset(), MPIPackBuffer::reset(), MPIUnpackBuffer::resize(), ApplicationInterface::restartFileFlag, ParallelLibrary::waitall(), ParallelLibrary::waitsome(), and Dakota::write_restart.
Referenced by ApplicationInterface::synch().
void static_schedule_evaluations | ( | ) | [private] |
blocking static schedule of all evaluations in beforeSynchCorePRPQueue using message passing; executes on iteratorComm master
This code runs on the iteratorCommRank 0 processor (the iterator) and is called from synch() in order to assign a static schedule. It matches serve_evaluations_peer() for any other processors within the 1st evaluation partition and serve_evaluations_synch()/serve_evaluations_asynch() for all other evaluation partitions (depending on asynchLocalEvalConcurrency). It performs function evaluations locally for its portion of the static schedule using either asynchronous_local_evaluations() or synchronous_local_evaluations(). Single-level and multilevel parallel use intra- and inter-communicators, respectively, for send/receive. Specific syntax is encapsulated within ParallelLibrary. The iteratorCommRank 0 processor assigns the static schedule since it is the only processor with access to beforeSynchCorePRPQueue (it runs the iterator and calls synchronize). The alternate design of each peer selecting its own jobs using the modulus operator would be applicable if execution of this function (and therefore the job list) were distributed.
References ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::asynchLocalEvalStatic, ApplicationInterface::asynchronous_local_evaluations(), ApplicationInterface::beforeSynchCorePRPQueue, Dakota::data_pairs, ApplicationInterface::evalCacheFlag, ParallelLibrary::free(), ParallelLibrary::irecv_ie(), ParallelLibrary::isend_ie(), ApplicationInterface::lenResponseMessage, ApplicationInterface::numEvalServers, Interface::outputLevel, ApplicationInterface::parallelLib, Interface::rawResponseMap, MPIUnpackBuffer::resize(), ApplicationInterface::restartFileFlag, ApplicationInterface::synchronous_local_evaluations(), ParallelLibrary::waitall(), and Dakota::write_restart.
Referenced by ApplicationInterface::synch().
void asynchronous_local_evaluations | ( | PRPQueue & | prp_queue | ) | [private] |
perform all jobs in prp_queue using asynchronous approaches on the local processor
This function provides blocking synchronization for the local asynch case (background system call, nonblocking fork, or threads). It can be called from synch() for a complete local scheduling of all asynchronous jobs or from static_schedule_evaluations() to perform a local portion of the total job set. It uses the derived_map_asynch() to initiate asynchronous evaluations and derived_synch() to capture completed jobs, and mirrors the self_schedule_evaluations() message passing scheduler as much as possible (derived_synch() is modeled after MPI_Waitsome()).
References ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::completionSet, Dakota::data_pairs, ApplicationInterface::derived_map_asynch(), ApplicationInterface::derived_synch(), ApplicationInterface::evalCacheFlag, Dakota::lookup_by_eval_id(), Interface::outputLevel, Interface::rawResponseMap, ApplicationInterface::restartFileFlag, and Dakota::write_restart.
Referenced by ApplicationInterface::static_schedule_evaluations(), and ApplicationInterface::synch().
void asynchronous_local_evaluations_static | ( | PRPQueue & | prp_queue | ) | [private] |
perform all the jobs in prp_queue using asynchronous approaches on the local processor, but schedule statically such that eval_id is always replaced with an equivalent one, modulo asynchLocalEvalConcurrency
Locally statically-scheduled counterpart to asynchronous_local_evaluations. This scheduling policy specifically ensures that a completed asynchronous evaluation eval_id is replaced with an equivalent one, modulo asynchLocalEvalConcurrency. Designed to help with parallel tiling. A disadvantage of this scheduling policy is that is could leave local asynchronous worker "servers" idle in parsing the prp_queue, e.g., when restarting and some evals are already complete. In fact, anytime this function is called with non-contiguous eval_id's the full possible concurrency won't be leveraged.
This is currently only supported when DAKOTA is running in serial. Supporting in the MPI static / asynch local hybrid mode would require MPI static schedule that is either fully round-robin or fully block scheduled, not the present hybrid. It is not clear how to support this in the MPI self scheduled / asynch local hybrid mode.
If local evaluation concurrency is unlimited, this function is not needed.
References ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::completionSet, Dakota::data_pairs, ApplicationInterface::derived_map_asynch(), ApplicationInterface::derived_synch(), ApplicationInterface::evalCacheFlag, ApplicationInterface::localServerJobMap, Dakota::lookup_by_eval_id(), Interface::outputLevel, Interface::rawResponseMap, ApplicationInterface::restartFileFlag, ApplicationInterface::runningSet, and Dakota::write_restart.
Referenced by ApplicationInterface::synch().
void synchronous_local_evaluations | ( | PRPQueue & | prp_queue | ) | [private] |
perform all jobs in prp_queue using synchronous approaches on the local processor
This function provides blocking synchronization for the local synchronous case (foreground system call, blocking fork, or procedure call from derived_map()). It is called from static_schedule_evaluations() to perform a local portion of the total job set.
References ParallelLibrary::bcast_e(), Dakota::data_pairs, ApplicationInterface::derived_map(), ApplicationInterface::evalCacheFlag, ApplicationInterface::lenVarsActSetMessage, ApplicationInterface::manage_failure(), Interface::multiProcEvalFlag, Interface::outputLevel, ApplicationInterface::parallelLib, Interface::rawResponseMap, ApplicationInterface::restartFileFlag, and Dakota::write_restart.
Referenced by ApplicationInterface::static_schedule_evaluations().
void asynchronous_local_evaluations_nowait | ( | PRPQueue & | prp_queue | ) | [private] |
launch new jobs in prp_queue asynchronously (if capacity is available), perform nonblocking query of all running jobs, and process any completed jobs (handles both local self- and local static-scheduling cases)
This function provides nonblocking synchronization for the local asynch case (background system call, nonblocking fork, or threads). It is called from synch_nowait() and passed the complete set of all asynchronous jobs (beforeSynchCorePRPQueue). It uses derived_map_asynch() to initiate asynchronous evaluations and derived_synch_nowait() to capture completed jobs in nonblocking mode. It mirrors a nonblocking message passing scheduler as much as possible (derived_synch_nowait() modeled after MPI_Testsome()). The result of this function is rawResponseMap, which uses eval_id as a key. It is assumed that the incoming prp_queue contains only active and new jobs - i.e., all completed jobs are cleared by synch_nowait().
Also supports asynchronous local evaluations with static scheduling. This scheduling policy specifically ensures that a completed asynchronous evaluation eval_id is replaced with an equivalent one, modulo asynchLocalEvalConcurrency. In the nowait case, this could render some servers idle if evaluations don't come in eval_id order or some evaluations are cancelled by the caller in between calls. If this function is called with unlimited local eval concurrency, the static scheduling request is ignored.
References ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::asynchLocalEvalStatic, ApplicationInterface::completionSet, Dakota::data_pairs, ApplicationInterface::derived_map_asynch(), ApplicationInterface::derived_synch_nowait(), ApplicationInterface::evalCacheFlag, ApplicationInterface::localServerJobMap, Dakota::lookup_by_eval_id(), Interface::outputLevel, Interface::rawResponseMap, ApplicationInterface::restartFileFlag, ApplicationInterface::runningSet, and Dakota::write_restart.
Referenced by ApplicationInterface::synch_nowait().
void serve_evaluations_synch | ( | ) | [private] |
serve the evaluation message passing schedulers and perform one synchronous evaluation at a time
This code is invoked by serve_evaluations() to perform one synchronous job at a time on each slave/peer server. The servers receive requests (blocking receive), do local synchronous maps, and return results. This is done continuously until a termination signal is received from the master (sent via stop_evaluation_servers()).
References ParallelLibrary::bcast_e(), ApplicationInterface::derived_map(), ApplicationInterface::evalCommRank, ParallelLibrary::isend_ie(), ApplicationInterface::lenResponseMessage, ApplicationInterface::lenVarsActSetMessage, ApplicationInterface::manage_failure(), Interface::multiProcEvalFlag, ApplicationInterface::parallelLib, ParallelLibrary::recv_ie(), MPIPackBuffer::reset(), ParallelLibrary::wait(), and Variables::write_annotated().
Referenced by ApplicationInterface::serve_evaluations().
void serve_evaluations_asynch | ( | ) | [private] |
serve the evaluation message passing schedulers and manage multiple asynchronous evaluations
This code is invoked by serve_evaluations() to perform multiple asynchronous jobs on each slave/peer server. The servers test for any incoming jobs, launch any new jobs, process any completed jobs, and return any results. Each of these components is nonblocking, although the server loop continues until a termination signal is received from the master (sent via stop_evaluation_servers()). In the master-slave case, the master maintains the correct number of jobs on each slave. In the static scheduling case, each server is responsible for limiting concurrency (since the entire static schedule is sent to the peers at start up).
References Dakota::abort_handler(), ApplicationInterface::asynchLocalEvalConcurrency, ApplicationInterface::completionSet, ApplicationInterface::derived_map_asynch(), ApplicationInterface::derived_synch_nowait(), Interface::idInterface, ParallelLibrary::irecv_ie(), ApplicationInterface::lenResponseMessage, ApplicationInterface::lenVarsActSetMessage, Dakota::lookup_by_eval_id(), ApplicationInterface::parallelLib, ParallelLibrary::recv_ie(), MPIUnpackBuffer::reset(), ParallelLibrary::send_ie(), and ParallelLibrary::test().
Referenced by ApplicationInterface::serve_evaluations().
void serve_evaluations_peer | ( | ) | [private] |
serve the evaluation message passing schedulers and perform one synchronous evaluation at a time as part of the 1st peer
This code is invoked by serve_evaluations() to perform a synchronous evaluation in coordination with the iteratorCommRank 0 processor (the iterator) for static schedules. The bcast() matches either the bcast() in synchronous_local_evaluations(), which is invoked by static_schedule_evaluations()), or the bcast() in map().
References ParallelLibrary::bcast_e(), ApplicationInterface::derived_map(), ApplicationInterface::lenVarsActSetMessage, ApplicationInterface::manage_failure(), ApplicationInterface::parallelLib, and Variables::write_annotated().
Referenced by ApplicationInterface::serve_evaluations().