*** 25.13.2 features

- DAGMan :dag-cmd:`PARENT/CHILD` dependencies can now be declared *WEAK*,
  meaning a child node only requires its parent(s) to finish executing
  rather than to succeed. See :ref:`Weak Dependencies<DAG weak dependencies>`.
  :jira:`2690`

- Improve how an AP administrator can automatically provide OAuth
  credentials for all jobs.
  Configuration knob :macro:`SUBMIT_ADD_LOCAL_CREDMON_PROVIDERS` specifies
  whether the AP is providing credentials for all jobs (default is true).
  Configuration knob :macro:`SUBMIT_ADD_LOCAL_CREDMON_PROVIDER_NAMES`
  specifies which credential names are being provided for all jobs
  (default is all credentials that the AP can generate by itself).
  Remote submitters no longer need to alter their configuration files to
  submit to an AP that provides credentials for all jobs.
  :jira:`3391`

- When :macro:`SINGULARITY_TARGET_DIR` is true and file transfer is 
  off, we now mount the _CONDOR_SCRATCH_DIR on /condor_scratch, as 
  there was nowhere else to mount it.
  :jira:`3768`

- Add a helpful "how to monitor jobs" tip to stderr after
  :tool:`condor_submit` runs.
  :jira:`3173`

*** 25.13.2 bugs

- Fixed a bug where the *condor_credmon_oauth* would not create and/or
  refresh credentials containing a user-supplied handle. Consequently,
  underscores (and other special characters) are no longer allowed in
  credential provider names by default. This behavior change can be
  reverted by setting :macro:`CREDMON_ALLOW_SPECIAL_CHAR_NAMES` to true,
  though this will disable the use of user-supplied handles.
  See :ref:`provider_name_restrictions` for details.
  :jira:`2860`

- The Archive Librarian now properly reduces the database size
  on disk during garbage collection.
  :jira:`3826`

- Fixed a bug where the archive librarian could mistake a rotated history
  file for the still-active one after a daemon restart, if the restart
  landed before that file was fully read. This caused the file to never be
  marked as fully read, and if it was later removed from disk, it would be
  incorrectly renamed with a ``.REMOVED`` suffix and lose its recorded
  rotation date.
  :jira:`3830`

- The ClassAd functions :classad-function:`versionGT`,
  :classad-function:`versionGE`, :classad-function:`versionLT`,
  :classad-function:`versionLE` and :classad-function:`versionEQ` are now
  case-insensitive, matching all other ClassAd functions.
  :jira:`3818`

- :tool:`condor_history` now selects the schedd to query the same way
  :tool:`condor_submit` and :tool:`condor_q` do, honoring both
  :macro:`SCHEDD_HOST` and :macro:`SCHEDD_NAME`.  Previously only
  :macro:`SCHEDD_HOST` was consulted and :macro:`SCHEDD_NAME` was ignored. Use
  ``-local`` to force reading the local history file regardless of
  configuration.
  :jira:`3839`

- Fix a very rare collector crash when creating the CA certificates file
  the first time and a view collector was also configured to run.
  :jira:`3832`

- Fix a DAGMan crash that could occur while using common input transfers
  for jobs specified in a DAG.
  :jira:`3835`

*** 25.0.13 bugs

*** 25.12.2 features

- Introducing the :ref:`archive-librarian`. A new daemon that maintains an
  SQLite3 database of indexes to historical job records located in an Access
  Point's :macro:`HISTORY` archive.
  :jira:`3143`

- The archive librarian now stores the :ad-attr:`DAGManJobId`, :ad-attr:`JobBatchId`,
  and :ad-attr:`JobBatchName` of job records into the library database.
  This allows database queries at a DAG workflow level to be executed.
  :jira:`3799`

- :tool:`condor_userprio` ``-setfloor`` and ``-setfactor`` now accept a
  ``-duration`` option to install a time-limited *lease* on the floor or
  priority factor: the prior value is automatically restored after the
  given number of seconds. Leases are persistent across
  *condor_negotiator* restarts. New ``-cancelfloorlease`` and
  ``-cancelfactorlease`` options cancel an in-effect lease. This
  mirrors the existing ``-setceiling`` lease support.
  :jira:`3764`

- Scheduler universe jobs such as DAGMan will now go into cool down
  for :macro:`SCHEDULER_UNIVERSE_COOL_DOWN_DURATION` seconds when the
  *condor_schedd* fails to launch the job.
  :jira:`3769`

- The headers printed out by the :tool:`condor_top` are now multi-row
  and should be clearer to understand.
  :jira:`3247`

- The *condor_schedd* now writes an audit log to ``$(LOG)/SchedAuditLog``
  by default.  In previous versions of HTCondor this was disabled.
  :jira:`3770`

- :tool:`condor_ssh_to_job` now also sets the supplementary groups of the
  shell inside the container to whatever the job had.
  :jira:`3741`

- VM universe now supports the ``user`` type networking with the **passt**
  backend allowing outbound network connectivity on EL10 execution points.
  :jira:`3735`

- Honor the :subcom:`transfer_executable` submit command for **batch**
  grid universe jobs.
  :jira:`3772`

- The OneDrive file transfer plugin now supports uploading files and
  directories, and downloading entire directories recursively, in
  addition to downloading individual files. Large files are uploaded in
  chunks via an upload session. Thanks to Zach McGrew for this contribution.
  :jira:`3792`
- HTCondor Connection Brokering (CCB) now supports a streaming mode that
  brokers connections between two daemons that are *both* behind a CCB,
  without opening any incoming ports. When the requesting daemon is itself
  CCB-routed, the CCB server has the target daemon connect back to the CCB
  server and then relays bytes between the two connections; the end-to-end
  security handshake is still performed directly between the two daemons, so
  the CCB server never decrypts their traffic. Streaming mode is enabled by
  default and controlled by the new configuration knobs
  :macro:`CCB_SERVER_STREAMING` and :macro:`CCB_CLIENT_STREAMING`.
  :jira:`3805`

- Jobs that transfer input sandboxes now log a file transfer started 
  event even when there list of files to be transferred is empty.
  Note that the transfer finished event has always been logged
  for empty file lists.  
  :jira:`3777`

- The ``htcondor ep rehome`` command directs a *condor_startd* to evict its
  running jobs and direct-attach to a different *condor_schedd*. The new
  ``--reboot`` option additionally reboots the execution point host once all
  jobs have been evicted, persisting the direct-attach configuration so the
  host comes back attached to the new *condor_schedd*. The reboot is gated
  by :macro:`STARTD_REHOME_ALLOW_REBOOT` on the EP and uses
  :macro:`STARTD_REBOOT_COMMAND` (default ``/sbin/reboot``) to perform the
  reboot.
  :jira:`3766`

*** 25.12.2 bugs

- Fixed a bug that could cause file descriptor leaks in *blahpd*.
  :jira:`3786`

- Improved the interaction between the *condor_schedd* and the
  *condor_negotiator* when the :macro:`SCHEDD_INTERVAL` was set
  to a very short value.  Previously, short values (< 10 seconds)
  could cause very long delays in communications.
  :jira:`3751`
- Fixed a regression where :macro:`DC_DAEMON_LIST` would treat everything after
  the initial ``+`` as a single entry.  It again properly handles a list of
  daemon names after the initial ``+``. And to avoid future problems the
  initial ``+`` is now optional; Built-in daemons can no longer be removed
  as DC daemons using this config variable.
  :jira:`3845`

*** 25.0.12 bugs

*** 25.11.1 bugs

- *Security Item*: This release of HTCondor fixes a security-related bug described at

  - `http://htcondor.org/security/vulnerabilities/HTCONDOR-2026-0001 <http://htcondor.org/security/vulnerabilities/HTCONDOR-2026-0001>`_.

  :jira:`3808`

- Fixed a bug that could cause the *condor_negotiator* accountant log
  (``Accountantnew.log``) to grow without bound.  Records for slots that
  were no longer claimed were no longer being removed during each
  negotiation cycle, so they accumulated indefinitely.  This bug was
  introduced in HTCondor 25.11.0.
  :jira:`3795`

*** 25.11.0 features

- The *condor_schedd* will now prevent a single user from monopolizing the input transfer
  queue and starving other user jobs from receiving matches by preventing a user from getting
  new matches when using more than 20% of the input transfer queue. This can be disabled
  by configuring :macro:`MAX_CONCURRENT_UPLOADS_PER_USER` to 0.
  :jira:`3664`

- Updated :tool:`condor_watch_q` to automatically group jobs by :ad-attr:`JobBatchName`
  correctly for jobs submitted after the tool began executing. The incorrect
  behavior was noticeable when using :tool:`condor_watch_q` to track a DAGMan
  execution.
  :jira:`3656`

- :tool:`condor_dag_checker` now has options to process and check the existence
  and validity of node submit submit descriptions, Sub-DAG files, and scripts.
  :jira:`3540`

- Added the ability for the *condor_gangliad* to aggregate the derivatives (deltas) of ClassAd attributes. This is useful
  to monitor numeric attributes that are "counters", i.e. attributes that start at zero and monotonically increase
  throughout the lifetime of a daemon.
  :jira:`3687`

- :tool:`condor_config_val` now accepts a ``-trace`` option which prints the file, line and previous value
  each time a value is replaced as the configuration files are read.
  :jira:`3699`

- Added new ``-duration`` option to :tool:`condor_userprio` to create a persistent
  time-limited *lease* for ceiling values set with ``-setceiling``. The prior
  ceiling value is automatically restored when after the lease expires or is
  canceled with the ``-cancelceilinglease`` option.
  :jira:`3518`

- Scheduler universe jobs (i.e. *condor_dagman*) no longer have their **argv[0]** set
  to ``condor_scheduniv_exec.<cluster>.<proc>`` preserving the executable name.
  This allows 'multibin' programs, common in Golang and Rust, that rely on checking
  **argv[0]** to function correctly in the scheduler universe.
  :jira:`3711`

- :tool:`condor_status` now accepts a ``-health`` option which displays machines
  that have less than 100% health, and totals for machines by health.
  :jira:`3714`
  
- The OAuth credmon web server now respects :macro:`CREDMON_WEB_PREFIX`. This is
  useful if the credmon web server is running under a different host name.
  :jira:`3746`

- :tool:`condor_ssh_to_job` now works for docker universe or container universe jobs
  when the path-name of the :macro:`EXECUTE` directory is longer than about 80 characters.
  :jira:`3729`

- User supplied transfer plugins can now be conditionally disabled in the
  *condor_starter* using a new configuration variable.
  :macro:`STARTER_DISABLE_USER_SUPPLIED_TRANSFER_PLUGINS`.
  :jira:`3722`

- Docker universe jobs that require refreshed credentials now immediately
  go on hold after exceeding their :subcom:`request_memory` requirements
  rather than being delayed for 20 minutes.
  :jira:`3672`

- DAGMan will no longer re-attempt placement of jobs to the *condor_schedd*
  when submission will never work such as when an invalid job description file
  is provided to a node. Note, this only occurs when DAGMan is in direct
  submission mode (default).
  :jira:`3716`

- Changed the default value of :macro:`QUEUE_CLEAN_INTERVAL` to 28800
  (8 hours).
  :jira:`3706`

- Added new ``"Cleaning"`` activity for claimed slots representing
  the time between the *condor_starter*\'s final update and actual reaping.
  During ``Claimed/Cleaning`` the slot will not match new work and all
  preempting requests are recorded to be applied after the *condor_starter* is
  reaped. The new :ad-attr:`TotalTimeClaimedCleaning` attribute accumulates
  the time spent in this activity.
  :jira:`3686`

- Slots used by an AP to transfer and store common files now have
  the prefix ``data`` rather than ``slot`` (e.g. ``data1_1``).  These
  slots also set the attribute :ad-attr:`IsDataSlot` in their slot ads.
  :jira:`3715`

*** 25.11.0 bugs

- Fix a recently introduced bug where a parallel universe job that cannot activate one
  of its claims can cause the *condor_schedd* to crash.
  :jira:`3704`

- Evaluating ClassAd expressions of the form ``list_of_ads.attribute_name``
  in :tool:`classad_eval` no longer causes a segmentation fault.
  :jira:`3554`

*** 25.0.11 bugs

*** 25.10.1 features

- Added ability to set DAGMan throttles via :tool:`htcondor dag throttle` and
  :meth:`htcondor2.DAGMan.throttle` while DAGMan is executing.
  :jira:`3584`

- DAGMan is now born with a security session from its parent *condor_schedd*
  allowing it to communicate without re-authentication.
  :jira:`3298`

- Scheduler universe jobs, such as *condor_dagman*, will now enter job
  cool down when they exit with a non-zero exit code. The duration of the
  cool down in controlled by :macro:`SCHEDULER_UNIVERSE_COOL_DOWN_DURATION`.
  :jira:`3067`

- Added new :tool:`htcondor dag resources` command, which shows
  the resource usage for running and historical jobs.
  :jira:`3636`

- Add new submit command :subcom:`retry_request_disk`, which operates
  like :subcom:`retry_request_memory` but for the ``Disk`` resource.
  :jira:`3552`

- When :macro:`STARTER_NESTED_SCRATCH` is true (the default), the **.chirp.config**
  file no longer pollutes the user's scratch directory, it is located
  in the slot's "user" directory.  The ``_CONDOR_CHIRP_CONFIG`` environment
  variable still points to the correct place.
  :jira:`2949`

- On cgroup v1 systems, match cgroup v2 behavior of disabling virtual
  memory swap space for jobs.  This can be disabled by setting
  :macro:`DISABLE_SWAP_FOR_JOB` to false.
  :jira:`3630`

- :tool:`condor_qusers`, :tool:`condor_status`, and :tool:`condor_who` now supports an `-aaf` option, for append
  autoformat, which displays columns like the `-af` option does, but
  appends them to the default output.
  :jira:`3604,3684`

- Container universe jobs whose image is an url (like ``docker://``) now work
  when file transfer is off.
  :jira:`3638`

- **batch** grid universe jobs are now run in the :ad-attr:`Iwd` or
  :ad-attr:`RemoteIwd` directory by default. Before, they were run in a
  temporary directory under the :ad-attr:`Iwd` or :ad-attr:`RemoteIwd`
  directory.
  The old behavior can be restored by setting ``blah_run_in_workdir=no``
  in the ``blah.config`` file.
  :jira:`2415`

- Improved the *blahpd*'s efficiency in monitoring job status.
  :jira:`3634,3640`

- Improved handling of OS usernames that include the ``@`` character.
  :jira:`3655`

- Added new submit variable ``$(JobListId)`` as an alternate name for ``$(ClusterId)``.
  :jira:`3586`

- Add new verb, **claims** to the :tool:`htcondor ap claims` cli tool.
  :jira:`3682`

- Improved the error handling and log messages in the rare case that a
  job fails after the *condor_starter* calls the ``fork()`` system call
  but before the ``exec()`` system call.
  :jira:`3654`

- Make :macro:`CHECK_REACTIVATE_AFTER_CHECKPOINT` true by default.
  :jira:`3564`

- Change value of :macro:`USERPRIO_USE_NEGOTIATOR_MODULAR_QUERY` to true.
  This allows the :tool:`condor_userprio` to use an interface to the negotiator
  to fetch accounting ads in a way that allows constraint and projection.
  This interface did not exist in pre htcondor 9.1 negotiators, and should
  be set to false for new :tool:`condor_userprio` to work with such old
  negotiators.
  :jira:`3649`

- Added new job ad attribute :ad-attr:`SubmitVersion`.  This string valued
  attribute is the condor version of the :tool:`condor_submit` version or
  python bindings version used to submit the job.  In the case of a remote 
  submit, it may be different than the :ad-attr:`CondorVersion[type=Schedd]` of ad,
  which represents the version of the *condor_schedd* the job was placed in.
  :jira:`3413`

- Fixed python deprecation warning in :tool:`condor_top`, so that it
  will run without errors on not-yet-released future version of python.
  :jira:`3643`

- The python method :meth:`htcondor2.Schedd.history` has an upper bound on the
  size in bytes of the constraint and projection argument.  Now return an
  clear error message when these bounds are exceeded.
  :jira:`3644` 

- Improve :tool:`condor_userprio` to return information about a specific
  user provided on the command line whether or not the user is actively
  running jobs and utilizing pool resources.
  :jira:`3283`

- Fixed a spurious error message in the :macro:`STARTER_LOG` about unhandled
  job exit.
  :jira:`3653`

*** 25.10.1 bugs

*** 25.0.10 features

- Add Ubuntu 26.04 (Resolute Raccoon) support.
  :jira:`3703`

*** 25.0.10 bugs

*** 25.8.2 features

- HTCondor now correctly detects and advertises the NVIDIA driver version
  for NVIDIA GPUs which are configured to run in "MIG" mode.
  :jira:`3626`

- Singularity or Apptainer jobs now also pass ``--rocm`` to GPU jobs
  to support AMD jobs, unless :macro:`SINGULARITY_ADD_ROCM_FLAG` is
  set to false.
  :jira:`3537`

- Added a new ``-transfer`` option to :tool:`condor_rm`.  When used to remove
  a running job, the job will transfer any output files it has produced
  back to the submit machine before exiting.
  :jira:`3519`

- :tool:`condor_q` now supports a ``-aaf`` option
  that works like ``-autoformat`` (``-af``) but appends the specified
  attributes as additional columns after the standard output columns,
  rather than replacing them.
  :jira:`3571`

- :tool:`condor_top` now supports a ``-history`` option to read the most
  recently recorded on disk performance statistics.  Runs much faster,
  and works even when the *condor_schedd* is slow or not running.
  :jira:`3582`

- :tool:`htcondor dag status` now also shows information about jobs
  transferring sandboxes and in the transfer queue.
  :jira:`3517`

- :tool:`htcondor job status` and :tool:`condor_q` now indicate when a job is
  in the cooldown state.
  :jira:`3525`

- :tool:`condor_run` no longer passes :jdl:`getenv = true` to the
  generated submit file.  There is a new option ``-getenv`` to 
  allow this.
  :jira:`3549`

- :tool:`condor_ssh_to_job` now work with container universe jobs where
  the Singularity or Apptainer runtime uses the setuid flow.
  :jira:`3308`

- The negotiator now has access to the :ad-attr:`RemoteUserFloor` to avoid
  going beneath a user's floor number of resources while preempting.
  :jira:`3226`

- The *condor_shadow* can now log a CSV record to the :macro:`SHADOW_RECONNECT_LOG`
  file each time it successfully reconnects to a disconnected starter or
  gives up trying.  Each record contains timing information about when the
  job was activated, the last known contact time, whether the reconnect
  succeeded, the configured lease duration, and whether the starter is
  known to be dead.  This is controlled by :macro:`SHADOW_LOG_RECONNECT`
  which defaults to ``True``.
  :jira:`3593`

- :tool:`condor_config_val` now has a -default option to shows the default
  value of a knob.
  :jira:`2683`

- Jobs that were submitted with spooled data files and have completed will
  now leave the queue after a successful retrieval of their output files
  (e.g. with :tool:`condor_transfer_data`).
  :jira:`3508`

- Failed URL transfer where the URL was longer than about 100 bytes would
  never have the URL scheme logged in the hold message or user log.
  :jira:`3412`

- All of the :tool:`htcondor` sub-tools (which are written in Python)
  support python 3.12.
  :jira:`2213`

- In the python bindings, functions arguments that used to have a default
  of the :obj:`None` type now take a python Optional.  This is more compliant
  with PEP 484, and is more in line with what python linting tools
  expect.  There should be no changes needed for clients of the python
  bindings.
  :jira:`3553`

- When cgroups are active, *condor_startds* now advertise
  :ad-attr:`CgroupEnforced` to be true.
  :jira:`3555`

- Enabling/disabling AP users now requires ADMINISTRATOR authorization.
  Previously, WRITE with queue superuser authorization was required.
  :jira:`3550`

- HTCondor now more quickly detects when the network connection between
  two daemons is broken or disconnected.
  :jira:`3547`

- Updated list of default environment variables that *condor_dagman* always
  grabs from the users environment to **not** include ``XDG_RUNTIME_DIR``.
  :jira:`1913`

- :tool:`condor_status` ``-server`` now correctly labels the count of discovered Slots
  in the summary table.
  :jira:`3535`

- When placing a job with credentials, the client will now find the
  address of the *condor_credd* via the *condor_schedd* when possible.
  The client already did this for Kerberos credentials; it now does so
  for all job credential types.
  :jira:`3536`

- Added new ``histogram`` option to :tool:`htcondor eventlog` and :tool:`htcondor dag`.
  :jira:`3530`

- The **copy_to_spool** command in :tool:`condor_submit` is no longer
  supported. This does not affect the ability to spool a job's full input sandbox.
  :jira:`3545`

- No longer warn about configuration knobs with dots in the names if the knob
  is group quota related, and begins with GROUP.
  :jira:`3544`

- The old ClassAd syntax is no longer supported in job transforms.
  :jira:`3538`

- Improved DAGMan debugging when to print warnings produced by *condor_schedd*\'s
  :ref:`submit-requirements`.
  :jira:`3573`

- Removed deprecated full rescue file for DAGMan. With this comes the removal of
  the configuration option ``DAGMAN_WRITE_PARTIAL_RESCUE`` and the command line
  option ``-DumpRescue`` for :tool:`condor_submit_dag`.  This has been deprecated since
  2011, so there should be no problems with compatibility.
  :jira:`3565`

- Improved DAGMan's debugging error when the provisioner node exits prior to notifying
  DAGMan of successful resource provisioning.
  :jira:`3542`

- Added new configuration parameter :macro:`SEC_USE_LOW_DATA_MODE`.
  When set to ``True``, daemons will reject large data messages from
  unauthenticated clients.
  The default is ``False``.
  :jira:`3393`

- The :macro:`VM_NETWORKING_TYPE` knob is now case insensitive,
  matching the documentation.
  :jira:`3557`

- Bootstrap CSS framework used in OAuth Credmon login page updated
  to version 5.3.8.
  :jira:`3576`

*** 25.8.2 bugs

- Fixed bugs where a job would be placed on hold if there was a problem transferring
  files via Pelican (OSDF) or HTTP.  Now in several cases where the problem was the
  fault of HTCSS and not the fault of the job, the job will go back to idle and attempt
  to be rematched instead of going on hold.  Several additional VacateReasonSubCodes where
  added in support of this; see section :ref:`Vacate Reason Codes`.
  :jira:`2952`

- Fixed a bug that caused the *condor_starter* to log confusing errors and not exit
  promptly if it failed to launch a Singularity job.
  :jira:`3531`

- Fixed a bug in container universe with :subcom:`container_target_dir` when file transfer
  was off, that caused jobs to start in the users home directory.
  :jira:`3575`

- Fixed a minor bug whereby the Access Point is unaware that the Execution Point relinquishes a
  match when the EP's :macro:`WANT_HOLD` is triggered.  This can result in the AP failing to
  activate the next job in a job list due to VacateReasonCode 1033 ``ActivationRefusedUnclaimed``.
  :jira:`3609`

*** 25.0.9 bugs

*** 25.7.2 features

- Jobs whose ClassAds are larger than 64 kilobytes long caused excessive load on the
  *condor_schedd*. The most common case of such a very large ClassAd is the use
  of :subcom:`getenv`.  This could cause the :ad-attr:`RecentDaemonCoreDutyCycle` to
  spike to near 100% for hours, and cause :tool:`condor_submit` to timeout and
  fail. Pipe buffers in the *condor_schedd* are now appropriately sized to fix
  this problem.
  :jira:`3507`

- Improved :tool:`condor_watch_q` to sort rows of data display rows with jobs still
  active in the queue before rows of jobs that have all exited the AP job queue.
  :jira:`3516`

- Added new command line options to :tool:`condor_watch_q` to control
  displaying rows of jobs that have all exited the AP job queue.
  :jira:`3516`

- Added ``-interval`` command line option to :tool:`condor_watch_q` to control the
  refresh rate of the tool.
  :jira:`3528`

- When a job exceeds its memory request, instead of reporting the memory 
  measured in the last polling period, the error message reports the 
  size of the provisioned memory to more accurately report what happened.
  :jira:`3514`

- All file transfer records written to the :macro:`JOB_EPOCH_HISTORY` archive
  will contain :ad-attr:`Owner` and :ad-attr:`RemoteHost` copied from the associated
  job record.
  :jira:`3497`

- Added new :meth:`htcondor2.Startd.vacateSlot` and :meth:`htcondor2.Startd.vacate`
  methods to the python API to do equivalent actions as :tool:`condor_vacate`.
  :jira:`3449`

- Added support for submitting jobs to the Flux job management system via the
  grid universe **batch** type.
  :jira:`3500`

*** 25.7.2 bugs

*** 25.0.8 bugs

*** 25.6.1 features

- DAGMan now uses the new DAG file parser introduced for :tool:`condor_dag_checker`. The old
  parser will be deprecated in the future but can be used by setting :macro:`DAGMAN_USE_OLD_FILE_PARSER`
  to ``True``.
  :jira:`3304`

- Augment the *condor_kbdd* to use the systemd sd-login library to find user
  sessions, since /usr/run/utmp is no longer available on newer platforms.
  Also, add idle check support for Mutter(Gnome), which works on both
  Wayland and X11.
  :jira:`3452`

- Added new ``-RescueFile`` option to :tool:`condor_submit_dag` to specify a
  DAGMan rescue file by name for state restoration.
  :jira:`3425`

- The submit knob :macro:`SUBMIT_REQUEST_MISSING_UNITS` now also applies to
  :subcom:`retry_request_memory` as well.
  :jira:`3419`

- Added :meth:`~classad2.ClassAd.formatJson` and :meth:`~classad2.ClassAd.formatOld` as
  aliases for :meth:`~classad2.ClassAd.printJson` and :meth:`~classad2.ClassAd.printOld`,
  respectively.
  :jira:`3341`

- If a docker universe jobs fails on an execution point because the image
  was built for a machine architecture that does not match the execution
  point, the hold message now more clearly indicates why.
  :jira:`3447`

- When the attempt to claim a slot to run a job is refused, the reason for the
  refusal is now reported back to the *condor_shadow* which logs the reason and
  adds the reason code to the :ad-attr:`NumVacatesByReason` attribute of the job.
  :jira:`3388`

- The option for the *condor_schedd* to keep job event log files open for
  a period of time has been removed.
  This was controlled by the now-removed configuration parameters
  USERLOG_FILE_CACHE_MAX and USERLOG_FILE_CACHE_CLEAR_INTERVAL.
  It was disabled by default.
  :jira:`3367`

- By default, HTCondor will now send extra information to "multi-file" file
  transfer plug-ins if such is specified in the job.  All such plug-ins
  should safely ignore this extra information.  To ease the transition if a
  plug-in does not, the extra information may be disabled by unsetting
  :macro:`ASSUME_COMPATIBLE_MULTIFILE_PLUGINS` on the relevant EPs.  (We
  have deliberately not documentation the format of the extra information
  because this feature is under active development.)

  Plug-in authors may find the
  :doc:`/apis/file-transfer-plugins/index` documentation helpful.
  :jira:`3357`

*** 25.6.1 bugs

- Fixed a bug when STARTER_NESTED_SCRATCH was true that broke the
  :subcom:`starter_log` submit command.
  :jira:`3408`

- The EPs produced by :tool:`htcondor annex` now include the additional
  configuration distributed with the HTCondor tarball, which should make
  them less sensitive to version changes.  Additionally, the default
  configuration for HTCondor now enables the Pelican (OSDF) file transfer
  plug-in.
  :jira:`3410`

- Fixed a bug where :tool:`condor_ssh_to_job` when running into a
  Singularity or Apptainer enabled job with :macro:`SINGULARITY_TARGET_DIR`
  set to true would start out with the current directory one higher than
  the job.
  :jira:`3467`

- Fixed a bug in the handling of :subcom:`retry_request_memory` where a
  list of values using a scale such as ``GB`` would generate an error
  or fail quietly if the ``B`` was not present.
  :jira:`3458`

- Fixed an intermittent bug where the *condor_shadow* would occasionally
  get stuck for 20 minutes trying to reconnect to a non-existent *condor_starter*
  when file transfer file fails for any reason.
  :jira:`3429`

*** 25.0.6 features

- Initial support for openSUSE Leap 16.
  :jira:`3233`

- Strip out VOMS and EC2 GAHP support in the HTCondor Python wheels, so that
  the HTCondor Python wheel is usable with the python-slim Docker image.
  :jira:`3469`

*** 25.0.6 bugs

*** 25.5.1 features

- A new configuration variable :macro:`FORCE_NEGOTIATOR_SLOT_WEIGHT` allows
  an administrator to configure the *condor_negotiator* so it uses its
  own expression of :macro:`SLOT_WEIGHT` rather than allowing the execution
  point to declare the weight of a slot.
  :jira:`3346`

- :macro:`LVM_CLEANUP_FAILURE_MAKES_BROKEN_SLOT` now defaults to ``True``\,
  causing failure to cleanup ephemeral Logical Volumes to break slot resources.
  If the logical volume is later successfully cleaned up then the associated
  broken resources will be restored.
  :jira:`3352`

- Added :ad-attr:`CleanupCategoryCounts` to record the number of cleanup
  reminders a *condor_startd* is tracking.
  :jira:`3352`

- The *condor_negotiator* will now apply the :macro:`REMOVE_SIGNIFICANT_ATTRIBUTES`
  list to the attributes it will be sending to the *condor_schedd* for resource requests.
  The *condor_negotiator* now also has a more complete set of known slot attributes which it
  will remove as well.  These changes should help make the *condor_schedd* more efficient when building
  resource requests for matchmaking.
  :jira:`3358`

- EP administrators may now alter the semantics of self-checkpointing jobs.
  After a job self-checkpoints and the checkpoint is uploaded, if
  :macro:`CHECK_REACTIVATE_AFTER_CHECKPOINT` is set, the EP will check if the slot could
  be reactivated then the job is restarted: otherwise the job is evicted with a
  :doc:`vacate reason <../codes-other-values/vacate-reason-codes>`.
  :jira:`3222`

- Fix up the *condor_credmon_oauth* to support Python 3.14.  With this, we believe that
  all of HTCondor will work with Python 3.14.
  :jira:`3365`

- Added configuration variables :macro:`JOB_ROUTER_SCHEDD1_ADDRESS_FILE` and
  :macro:`JOB_ROUTER_SCHEDD2_ADDRESS_FILE`. When configured, the *condor_job_router*
  will first use the address file to locate the source or destination *condor_schedd* and
  only query the *condor_collector* if the file does not contain an address. Use of these
  configuration variables improves the efficiency of the job router and eliminates
  one source of failure when routing.
  :jira:`1736`

- Removed the ``NAMED_CHROOT`` configuration parameter for the
  *condor_starter*. It was an inferior form of containerization that was
  never fully supported.
  :jira:`3380`

- Improved DAGMan's debugging for executed scripts.
  :jira:`3381`

*** 25.5.1 bugs

*** 25.0.5 features

- Initial support for Ubuntu 24.04 (noble numbat) on the arm64 platform.
  :jira:`3414`

*** 25.0.5 bugs

*** 25.4.0 features

- Changed the default value of :macro:`STARTER_NESTED_SCRATCH` to true.  When false,
  the job scratch directory (named ``dir_XXX``) is the immediate child of the :macro:`EXECUTE`
  directory.  When true, the job's scratch directory is a sub-directory thereof, named
  ``scratch``, with the intention that all condor metadata files will be moved to peer
  directories, and not pollute the job's scratch directory with condor control files.
  :jira:`3080`

- The *condor_starter* can now refresh job credentials and respond to
  commands while doing output file transfers.
  :jira:`3259`

- When a job has a :subcom:`transfer_output_remaps` that maps to an
  absolute path, HTCondor will now create any missing intermediate
  directories in that path, if allowed by the user's permissions.
  :jira:`3318`

- :tool:`condor_watch_q` will now display for total number of jobs for
  clusters using late materialization.
  :jira:`3164`

- The *condor_schedd* can now limit the number of running jobs on a per-user basis.
  The *condor_schedd* will not allow more jobs than the :ad-attr:`MaxJobsRunning` in
  the user record to begin running.
  :jira:`3325`

- Added new job ClassAd attributes :ad-attr:`NumVacatesPreExecution`
  and :ad-attr:`NumVacatesByReasonPreExecution`. These attributes provide
  counts about why a job was kicked off of an Execution Point before the
  job was actually spawned (i.e. due to failure to transfer input files,
  failure to setup the job environment, etc).
  :jira:`3339`

- The *condor_startd* will now provision :ad-attr:`Disk` to slots as a specific quantity in KiB
  rather than as a percentage of free disk space.  :ad-attr:`Disk` provisioned to dynamic slots
  will now be much closer in size to the request. Provisioning statements in the
  configuration files are no longer required to be expressed as a percentage, but a percentage
  will still work as before.  The *condor_startd* now reports both :ad-attr:`TotalDisk`
  and :ad-attr:`DetectedDisk` in each slot and in the daemon ad.
  :jira:`3302`

- Added new Python binding functions to reload the ClassAd usermaps and to load
  a specific usermap for use with the ``userMap`` ClassAd function.
  :jira:`3274`

- Docker universe jobs now correctly report CPU and memory usage when the
  Docker server is installed in a non-standard place, pointed at with
  the ``DOCKER_HOST`` environment variable, such as with a docker-in-docker setup.
  :jira:`3280`

- More accurately report the reason why a job's execution was interrupted
  (via the evict log event and :ad-attr:`VacateReason` job attribute).
  :jira:`3295`

- Removed option to have the *condor_schedd* send claim lease keep-alives.
  Now, the *condor_startd* always sends the keep-alives, which was the
  default.
  :jira:`3312`

- For **batch** grid :subcom:`universe` jobs, the ``PATH`` environment variable
  is now used by default to locate the tools of the destination batch scheduler.
  You can specify a specific directory where the tools are located by setting
  the appropriate binpath variable in the ``blah.config`` file (e.g. ``slurm_binpath=/opt/slurm/bin``).
  :jira:`3326`

*** 25.4.0 bugs

*** 25.3.1 bugs

*** 25.0.3 bugs

*** 25.2.1 features

- A job can now specify both an initial value for :subcom:`request_memory`
  and a retry with a larger memory request if the job uses more memory than the initial request.
  The default value for :subcom:`request_memory` and for :subcom:`request_disk`
  have been changed so that jobs will no longer default to trying to grow the request
  based on previous usage.
  :jira:`3227`

- Improved DAGMan to allow :dag-cmd:`SUBMIT-DESCRIPTION`\'s to be declared
  after nodes that reference the named inline submit description.
  :jira:`3230`

- DAGMan will now output a structured message for job submission failures
  to the debug log file. This message is formatted similarly to the read
  userlog job event message.
  :jira:`3257`

- Added new DAGMan Post :dag-cmd:`SCRIPT` macro to provide exit code for each
  job associated with a node. This is in the form of a comma separated list.
  :jira:`3245`

- The *condor_credmon_local* can now use RSA-based (RS256) keys to generate
  tokens by setting :macro:`LOCAL_CREDMON_PRIVATE_KEY_ALGORITHM` to ``RS256``.
  :jira:`3266`

- Added the ability to set a ClassAd expression in
  :macro:`LOCAL_CREDMON_AUTHZ_TEMPLATE_EXPR` to define scopes for tokens
  generated by the *condor_credmon_local*.
  :jira:`3261`

- Added new fine grained attributes :ad-attr:`NumPendingClaimsInQueue` and
  :ad-attr:`NumPendingClaimsInContact` to the *condor_schedd* record.
  :jira:`3125`

*** 25.2.1 bugs

- Fixed a problem where setting :macro:`MAX_FILE_TRANSFER_PLUGIN_LIFETIME`
  to longer than :macro:`STARTER_FILE_XFER_STALL_TIMEOUT` would prevent the
  logging of output for hung file-transfer plug-ins.
  :jira:`3202`

*** 25.0.2 bugs

- Fixed a bug causing some file-transfer plug-ins (box, gdrive, and onedrive)
  to fail immediately on start-up.
  :jira:`3317`

- Fixed bug where when :macro:`SINGULARITY_TARGET_DIR` was set and
  :macro:`STARTER_NESTED_SCRATCH` was also set, then some environment
  variables that pointed into the job's execute directory were not correct.
  :jira:`3265`

*** 25.1.0 features

*** 25.1.0 bugs

*** 25.0.1 features

- New implementations of the Python bindings, ``htcondor2`` and ``classad2``. This
  re-implementation of the Python bindings fixes short-comings in the original
  implementation and eases future maintenance.
  :jira:`2153`

- The original implementation of the Python bindings has been removed.
  To use the new Python bindings, please read the
  :doc:`migration guide <../apis/python-bindings/api/version2/migration-guide>`.
  :jira:`3240`

*** 25.0.1 bugs

- Fixed a bug where *condor_qusers* would treat the ``add`` option as ``enable``
  when the leading dash was omitted.
  :jira:`3284`

