Bypass custom logic on import, and end every run with its total time

README.md

@@ -64,6 +64,9 @@ All three tools are driven by a **Shuffle definition** that follows `ShuffleDefi
 |-----------|------|---------|-------------|
 | `Timeout` | int | 2 | Minutes to wait for an **asynchronous solution import** to finish once the import job has started. It does not limit anything else. |
 | `StopOnError` | boolean | `false` | Stop the run at the first record or solution that fails, instead of logging it and carrying on |
+| `BypassSyncLogic` | boolean | `false` | Import without running synchronous plugins and workflows. See [Bypassing custom logic](#bypassing-custom-logic) |
+| `BypassAsyncLogic` | boolean | `false` | Import without running asynchronous plugins and workflows (Dataverse only) |
+| `BypassFlows` | boolean | `false` | Import without triggering Power Automate flows (Dataverse only) |
 
 Contains a `<Blocks>` element holding any combination of `<SolutionBlock>` and `<DataBlock>`,
 processed in order.
@@ -255,12 +258,47 @@ creates each record and updates it if it already exists.
 
 ---
 
+## Bypassing custom logic
+
+A data load into a configured environment fires every plugin, workflow and flow registered on
+the tables it writes to - once per record. That makes loads slow, fills the async queue, and can
+change the data on its way in: auto-numbering, validation, flows that send mail. Three attributes
+on `<ShuffleDefinition>` switch that off for the whole import:
+
+```xml
+<ShuffleDefinition BypassSyncLogic="true" BypassAsyncLogic="true" BypassFlows="true">
+```
+
+| Attribute | Skips | Sent as |
+|-----------|-------|---------|
+| `BypassSyncLogic` | Synchronous plugins and real-time workflows | `BypassBusinessLogicExecution = CustomSync` |
+| `BypassAsyncLogic` | Asynchronous plugins and background workflows | `BypassBusinessLogicExecution = CustomAsync` |
+| `BypassFlows` | Power Automate flows triggered by Dataverse | `SuppressCallbackRegistrationExpanderJob = true` |
+
+- **Every write in every data block** carries the parameters: batched and single creates and
+  updates, upserts, deletes, state and owner changes (also the deferred pass) and N:N
+  associations. Solution blocks are not affected.
+- **The user needs the `prvBypassCustomBusinessLogic` privilege** for sync and async logic
+  (System Administrators have it). Without it, the first write fails with Dataverse's error.
+- Microsoft's own logic still runs; only custom logic is skipped.
+- **On-premises 9.0 and 9.1** only support bypassing sync logic, sent as the older
+  `BypassCustomPluginExecution`; older versions cannot bypass at all. What a server cannot
+  bypass runs as usual: the import logs a warning and carries on, for example
+  `WARNING: This server (9.1.0.0) cannot bypass async plugins and workflows or Power Automate flows. They run as usual during this import.`
+  One definition therefore works for online and on-premises alike. The Builder's tooltips say the same.
+
+See Microsoft's [Bypass custom business logic](https://learn.microsoft.com/power-apps/developer/data-platform/bypass-custom-business-logic)
+and [Bypass Power Automate flows](https://learn.microsoft.com/power-apps/developer/data-platform/bypass-power-automate-flows).
+
+---
+
 ## What's new since 1.2023.5
 
 ### Import
 - **Opt-in batching** with `BatchSize`, using `CreateMultiple`/`UpdateMultiple`, `ExecuteMultipleRequest` or individual requests depending on what the environment supports
 - **`DeferStateAndOwner`** for batching records that carry state or owner
 - **Upsert path** for blocks matching on the primary key, with `UpsertMultiple`
+- **Bypass custom logic**: `BypassSyncLogic`, `BypassAsyncLogic` and `BypassFlows` on the definition skip plugins, workflows and flows during the import, with three matching checkboxes on the Builder's root node
 - The Builder has a **Batch size** field and a **Defer state and owner** checkbox on the Import node
 - **`PreRetrieveAll` reads the whole target table.** It stopped at 5000 records, so every source record whose match lay beyond that was created a second time
 - **`PreRetrieveAll` matching is a lookup** instead of a scan of the whole table for every record
@@ -283,6 +321,7 @@ creates each record and updates it if it already exists.
 - Log sections are nested correctly, also after a failed record or solution
 - Log files are named after the definition, operation, connection and time
 - The Runner no longer makes the import wait for its window to repaint, which cost about 15 ms per record
+- Every import and export ends with its total run time and counts, for example `Import finished in 1 min 58 s: 14970 created, 0 updated, 0 skipped, 0 deleted, 0 failed` - in the log, the Runner and the pipeline output
 
 ### Export
 - **Exports are complete.** Every export query stopped at 5000 records, so larger tables were cut off without a warning; all three - filter, FetchXML and intersect - now read every page

XTB/ShuffleBuilder.nuspec

@@ -23,7 +23,8 @@ Changes since 1.2023.5:
 
 - New Batch size field on the Import node (default 1, no batching; max 1000).
 - New Defer state and owner checkbox on the Import node, with a tooltip explaining when it helps.
-- Definitions follow the updated schema: BatchSize and DeferStateAndOwner on Import.
+- New Bypass sync logic, Bypass async logic and Bypass flows checkboxes on the ShuffleDefinition node.
+- Definitions follow the updated schema: BatchSize and DeferStateAndOwner on Import; BypassSyncLogic, BypassAsyncLogic and BypassFlows on ShuffleDefinition.
 - .NET Framework 4.8; ILMerge removed; DotNetZip replaced with System.IO.Compression.
 
 The import engine changes are listed in the Shuffle Runner release notes.

XTB/ShuffleDeployer.nuspec

@@ -31,6 +31,8 @@ Import
 - Multi-select choice columns (OptionSetValueCollection) export and import correctly.
 - Floating point (Double) columns can be imported from every format, not only Full.
 - Records with date columns are now skipped as identical when nothing changed, and a Match on a date column finds the existing record instead of creating a duplicate.
+- Bypass custom logic: BypassSyncLogic, BypassAsyncLogic and BypassFlows on the definition import without running plugins, workflows and Power Automate flows.
+- Every import and export ends with its total run time and counts.
 - An N:N relationship from a table to itself imports; every record used to fail.
 
 Logs

XTB/ShuffleRunner.nuspec

@@ -31,6 +31,8 @@ Import
 - Multi-select choice columns (OptionSetValueCollection) export and import correctly.
 - Floating point (Double) columns can be imported from every format, not only Full.
 - Records with date columns are now skipped as identical when nothing changed, and a Match on a date column finds the existing record instead of creating a duplicate.
+- Bypass custom logic: BypassSyncLogic, BypassAsyncLogic and BypassFlows on the definition import without running plugins, workflows and Power Automate flows.
+- Every import and export ends with its total run time and counts.
 - An N:N relationship from a table to itself imports; every record used to fail.
 
 Runner

shared/Xrm.Shuffle.Core/BypassContainer.cs

@@ -0,0 +1,26 @@
+namespace Cinteros.Crm.Utils.Shuffle
+{
+    using global::Xrm.Utils.Core.Common.Interfaces;
+    using Microsoft.Xrm.Sdk;
+
+    /// <summary>
+    /// The import's container with its service swapped for one that bypasses custom logic. The
+    /// log and values are shared, so the run reads as one.
+    /// </summary>
+    public sealed class BypassContainer : IExecutionContainer
+    {
+        private readonly IExecutionContainer inner;
+
+        public BypassContainer(IExecutionContainer inner, IOrganizationService service)
+        {
+            this.inner = inner;
+            Service = service;
+        }
+
+        public dynamic Values => inner.Values;
+
+        public ILoggable Logger => inner.Logger;
+
+        public IOrganizationService Service { get; }
+    }
+}