Repository navigation
Splunk 10.x Pods do not start - mongodb migration issue #1875
Description
We encountered the exact issue as described in #1489
Our environment is K8S open source.
Has there been any news whether the issue has been or will be fixed?
Below are the logs:
$ kubectl exec -n splunk -it tsc-standalone-0 -- cat /opt/splunk/var/log/splunk/migration.log.2026-05-07.11-44-08
Defaulted container "splunk" out of: splunk, copy-apps (init)
Migrating to:
VERSION=10.0.2
BUILD=e2d18b4767e9
PRODUCT=splunk
PLATFORM=Linux-x86_64
-> Currently configured KVSTore database path="/opt/splunk/var/lib/splunk/kvstore"
-> Version file="/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile42" does not exists or not accessible!
-> Currently used KVSTore version=. Expected version=4.2 or version=7.0
CPU Vendor: AuthenticAMD
CPU Family: 25
CPU Model: 1
CPU Brand: (
AVX Support: Yes
SSE4.2 Support: Yes
AES-NI Support: Yes
-> isSupportedArchitecture=1
-> isKVstoreDisabled=0
-> isKVstoreDatabaseFolderExist=1
-> isKVstoreDiagnosticsFolderExist=1
-> isKVstoreVersionFileFolderExist=1
-> isKVstoreVersionFileFolderEmpty=0
-> isKVstoreVersionFileMatched=1
-> isKVstoreVersionFromBsonMatched=0
-> isSupportedArchitecture=1
- Active KVStore version upgrade precheck FAILED!
-- This check is to ensure that KVStore version 4.2 been in use.
-- In order to fix this failed check, re-install the previous Splunk version, and follow the KVStore upgrade documentation: https://docs.splunk.com/Documentation/Splunk/9.3.0/Admin/MigrateKVstore#Upgrade_KV_store_server_to_version_4.2 .
Some upgrade prechecks failed!
ERROR while running splunk-preinstall.
Activity
Chaithanyanallamothu commentedon Oct 24, 2026
@drnguyen I checked the related issue #1489 and went through the startup/liveness probe scripts,
configuration.go, and the upgrade validation logic inupgrade.go.The missing
versionFile42andisKVstoreVersionFromBsonMatched=0errors look similar to what was reported in #1489. There was some discussion there about startup probes interrupting KV Store migration, but I couldn't find anything that confirms that's happening here.From the operator code, the startup probe timings are configurable, while the upgrade validation mainly checks component dependencies and image versions.
Could you share which Splunk version you upgraded from and which operator version you're using? Also, do the pod events show any probe failures or container restarts around the time
splunk-preinstallfails?I also noticed Splunk Support was suggested in #1489. Was a support case opened for this issue, and did they find anything?
I'd like to understand whether the preinstall check is failing on its own or whether the container is being restarted during the upgrade.