job.scheduler
=============

.. py:module:: job.scheduler


Attributes
----------

.. autoapisummary::

   job.scheduler.JobProviderCallback
   job.scheduler.logger
   job.scheduler.DEFAULT_JOB_MAX_DURATION


Classes
-------

.. autoapisummary::

   job.scheduler.Scheduler


Module Contents
---------------

.. py:data:: JobProviderCallback

.. py:data:: logger

.. py:data:: DEFAULT_JOB_MAX_DURATION
   :value: 86400


.. py:class:: Scheduler(job_provider: JobProviderCallback, collect: CollectCallback | None = None, queues: dict[str, int] | None = None, tokens: int = 1, job_timeout: int = DEFAULT_JOB_MAX_DURATION)

   Handle parallel execution of interdependent jobs.


   .. py:attribute:: job_provider


   .. py:attribute:: job_timeout
      :value: 86400



   .. py:attribute:: active_jobs
      :type:  list[e3.job.Job]
      :value: []



   .. py:attribute:: queued_jobs
      :value: 0



   .. py:attribute:: all_jobs_queued
      :value: False



   .. py:attribute:: message_queue
      :type:  queue.Queue[Any]


   .. py:attribute:: dag
      :type:  e3.collection.dag.DAG | None
      :value: None



   .. py:attribute:: dag_iterator
      :type:  e3.collection.dag.DAGIterator | None
      :value: None



   .. py:attribute:: start_time
      :type:  datetime.datetime | None
      :value: None



   .. py:attribute:: stop_time
      :type:  datetime.datetime | None
      :value: None



   .. py:attribute:: max_active_jobs
      :value: 0



   .. py:attribute:: queues
      :type:  dict[str, list[tuple[int, int, e3.job.Job]]]


   .. py:attribute:: tokens


   .. py:attribute:: n_tokens
      :value: 0



   .. py:attribute:: global_queue_index
      :value: 1



   .. py:attribute:: slots


   .. py:method:: safe_collect(job: e3.job.Job) -> bool

      Protect call to collect.

      This ensures for job such as JobProcess that there are no calls to
      Run during collect. Main goal is to avoid leak of handles from collect
      to a process spawned by a Job. On Unixes consequences of such leak is
      more a security concern than an operational one. On Windows, this can
      lead easily to file locking issues and thus might cause crashes.



   .. py:method:: safe_job_provider(uid: str, data: Any, predecessors: FrozenSet[str], notify_end: collections.abc.Callable[[str], None]) -> e3.job.Job

      Protect call to job_provider.

      See safe_collect commment above.



   .. py:method:: simple_provider(job_class: type[e3.job.Job]) -> JobProviderCallback
      :classmethod:


      Return a simple provider based on a given Job class.

      :param job_class: a subclass of Job



   .. py:method:: init_state(dag: e3.collection.dag.DAG) -> None

      Reinitialize the scheduler state (internal function).

      :param dag: the dag representing the list of job to execute



   .. py:property:: is_finished
      :type: bool


      Check if all jobs have been executed (internal).

      :return: True if complete



   .. py:method:: log_state() -> None

      Log the current state of the scheduler (internal).



   .. py:method:: run(dag: e3.collection.dag.DAG) -> None

      Launch the scheduler.



   .. py:method:: push(job: e3.job.Job) -> None

      Push a job into a queue.



   .. py:method:: enqueue() -> None

      Push into the queues job that are ready (internal).



   .. py:method:: launch() -> None

      Launch next jobs in the queues (internal).



   .. py:method:: wait() -> None

      Wait for the end of an active job.



