ℹ️ this log lists user most visible changes
⚠️ to find all changes use changelog.txt file
📖 Explanation: https://gist.github.com/andry81/d278e6d129ca1af326eafb67470a2ae3
📜 Legend: 🛡️ - security; 🔧 - fixed; 🆕 ✨ - new; 📝 - changed; 🔀 - refactor
- 📝 changed: bash: board/accum-stats.sh, inpage/accum-downloads.sh: warning over a bot crawler protection in the phpbb
- 🆕 new: ✨ bash/github/init-basic-workflow.sh, bash/github/init-print-workflow.sh, print-*.sh:
CHANGELOG_FILE_LRenvironment variable to explicitly control changelog output file line return format
- 🆕 new: ✨ bash/github/init-yq-workflow.sh, accum-*.sh:
YAML_OUTPUT_LRenvironment variable to explicitly control yaml output file line return format
- 🔧 fixed:bash/github/accum-rate-limits.sh: json format update
- 🔧 fixed:bash/github/init-yq-workflow.sh: improved workaround for
Spurious comment line duplication in yq v4.50.1
- 🔧 fixed:bash/github/init-yq-workflow.sh: workaround for
Spurious comment line duplication in yq v4.50.1: mikefarah/yq#2578
- 🔧 fixed:bash/github/init-yq-workflow.sh:
ENABLE_YAML_PRINT_ALLdidn't print all
- 🔧 fixed:bash/github/init-diff-workflow.sh: missed to check
patchexistence
- 🆕 new: ✨ bash/github/init-*-workflow.sh: always print the path and version of used tools
- 🆕 new: ✨ bash/github/init-yq-workflow.sh:
ENABLE_YAML_PRINT_ALLvariable to enable all prints altogether
- 🔧 fixed: bash/github/init-yq-workflow.sh: missed to detect
jqexistence in case ofhttps://github.com/kislyuk/yqimplementation
- 🔧 fixed: bash/github/init-github-workflow.sh:
gh_eval_github_env_filefunction rewrite to ignore value last line return
- 🔧 fixed: bash:
tkl_include*correct nest level value - 🆕 new: ✨ bash: Added usage of
tkl_register_call/tkl_execute_callsand implemented postponed gh-workflow initialization, when the firstinit_*function calls only after the lastinit-*.shscript is included. This shall avoid recursed initialization execution ininit-*.shscripts, when one script is loaded, but another one is not yet. - 📝 changed: _externals/tacklelib/bash/tacklelib: merge with the latest changes
- 🔧 fixed: bash/github/init-basic-workflow.sh: completely removed
GITHUB_ENVvariables conditional load feature, because must always be loaded unconditionally, otherwise variables may be lost on the next shell line of the same GitHub Action step
- 🆕 new: ✨ bash/github/enable-print-buffering.sh: added script to explicitly enable print buffering
- 🆕 new: ✨ bash/github/utils.sh:
gh_unset_github_env_var,gh_push*_var_to_stack,gh_pop_var_from_stackto be able to use variables stack between GitHub Action steps - 📝 changed: bash/github/utils.sh:
gh_set_env_varnow does export a variable instead of declare becauseGITHUB_ENVvariables does export too - 🔀 refactor: bash/github/utils.sh:
gh_update_github_env_varrenamed togh_set_github_env_var
- 🔧 fixed: bash/github/init-print-workflow.sh: annotations print duplication
- 🔧 fixed: bash: UTC timezone for URLs in
GHWF_REPO_STORE_COMMITS_URL/GHWF_REPO_STATS_COMMITS_URLvariables
- 🔧 fixed: bash/github/print-*.sh: unchanged or empty changelog file
- 🔧 fixed: bash/github/accum-rate-limits.sh: json format update
- 🆕 new: ✨ bash: use
ENABLE_REPO_STORE_COMMITS_URL_PRINT_TO_CHANGELOG/ENABLE_REPO_STATS_COMMITS_URL_PRINT_TO_CHANGELOGandGHWF_REPO_STORE_COMMITS_URL/GHWF_REPO_STATS_COMMITS_URLvariables to printContent Store Repository/Statistic Output Repositorycommit URL into being committed changelog file
- 🔧 fixed: bash/github/accum-stats.sh: avoid update if nothing changed including 14d
countanduniquesdecrement by the last day
- 🔧 fixed: bash/github/accum-stats.sh: 14d
countanduniquesdecrement by the last day
- 🆕 new: ✨ bash/github/is-flag-true-in-flags-expr-string.sh: added
gh_is_flag_true_in_flags_expr_stringfunction to evaluate flag expression strings
- 🔧 fixed: bash/github/print-*.sh: multiline annotations print
- 🆕 new: ✨ bash/_common: added
update-permissions.shscript to update file permissions byfindpattern all together (ex:update-permissions.sh ug+x "$GH_WORKFLOW_ROOT" -iname '*.sh') - 🆕 new: ✨ bash: print GitHub Actions workflow run number in case of print workflow run URL (
ENABLE_GITHUB_ACTIONS_RUN_URL_PRINT_TO_CHANGELOG=1) - 📝 changed: bash/cache/accum-content.sh: export specific
GH_WORKFLOW_FLAGSenvironment variable into nested bash contexts
- 🆕 new: ✨ bash/cache/accum-content.sh: added
entries[0]/init/shellandentries[0]/init/runconfig content file fields to run entries initialization script - 🆕 new: ✨ bash/github/init-xq-workflow.sh: added
XQ_CMDLINE_READandXMLSTARLET_CMDLINE_SELvariable to be able to use different xml query implementations
- 🆕 new: ✨ bash/cache/accum-content.sh: added
download-validate/shellanddownload-validate/runconfig content file fields to run a file download validation script - 🆕 new: ✨ bash/github: added
init-xq-workflow.shscript to intialize workflow for thexqutility
- 🆕 new: ✨ bash: use
ENABLE_GITHUB_ACTIONS_RUN_URL_PRINT_TO_CHANGELOGandGHWF_GITHUB_ACTIONS_RUN_URLvariables to print GitHub Actions run URL into the changelog file
- 🆕 new: ✨ bash/github: added
eval-from-args.shwith--eval_github_envflag to evaluate set of commands and additionally does evaluateGITHUB_ENVfile before the execution
- 🆕 new: ✨ bash/cache/accum-content.sh: added
NO_DOWNLOAD_ENTRIESandNO_DOWNLOAD_ENTRIES_AND_CREATE_EMPTY_INSTEADvariables to be able to debug locally without download - 📝 changed: bash: board/accum-stats.sh, cache/accum-content.sh, inpage/accum-downloads.sh: print-*.sh: always print stderr unconditionally to a return code
- 📝 changed: bash/github/print-*.sh:
GHWF_PRINT_ERROR_BUF_STRvariable assign bygh_set_env_varto be able to debug usingGITHUB_ENVfile locally
- 🆕 new: ✨ bash/cache/accum-content.sh: added prev/next file size print
- 🆕 new: ✨ bash/cache: init-print-workflow.sh, accum-content.sh: enabled changed file notice messages groupping into a single annotation to fit the GitHub Actions summary maximum limit as 10 annotations (restored, because the message truncation actually was a message shrink into a single line, after expand it does restore back into multiline string)
- 🔧 fixed: bash/github: utils.sh, set-env-from-args.sh: missed to ignore
GITHUB_*builtin variables, execution fixup - 🆕 new: ✨ bash/github: init-basic-workflow.sh, enable-github-env-autoeval.sh: Ability to apply GitHub Actions job environment variables by the
GITHUB_ENVfile path automatically on each GitHub Actions job step line instead of on the next step only. A workflow does require to enable this explicitly using theenable-github-env-autoeval.shscript run. - 📝 changed: bash/github: utils.sh, set-env-from-args.sh: added global bash variables assign additionally to
GITHUB_ENVfile update to apply variables assign immediately for the same GitHub Actions job step line - 🔀 refactor: bash/github:
GH_*global environment variables renamed intoGHWF_*, added prefix to the rest
- 🔧 fixed: bash/github/print-*.sh: execution fixup
- 🆕 new: ✨ bash/github: init-print-workflow.sh, print-*.sh, *-print-annotation-group.sh: ability to group sequence of prints with the same annotation type into single annotation to prevent too many annotations because of the limit in the GitHub Actions job summary (currently is 10)
- 🆕 new: ✨ bash/github/accum-rate-limits.sh: added usage of a print annotation group
- 🔧 fixed: bash/github/print-*.sh: execution fixup
- 🆕 new: ✨ bash/github: init-print-workflow.sh, print-*.sh, flush-print-annotation.shs: print annotations buffering support
- 🆕 new: ✨ bash/github: init-basic-workflow.sh, print-*.sh:
CHANGELOG_BUF_STRupdate between GitHub pipeline steps
- 🔧 fixed: bash/github/print-*.sh: notices, warnings, errors functions fixup
- 🔧 fixed: bash/github/accum-rate-limits.sh: json format change fixup
- 📝 changed: bash/github/accum-rate-limits.sh: changed rate limits json format change message from an error to a warning
- 🆕 new: ✨ bash/github/init-yq-workflow.sh: added
ENABLE_YAML_DIFF_PRINT_BEFORE_PATCH,ENABLE_YAML_PRINT_AFTER_PATCHvariables to print diff/yaml file before patch instead of after edit (has priority overENABLE_YAML_DIFF_PRINT_AFTER_EDIT,ENABLE_YAML_PRINT_AFTER_EDITvariables)
- 🔧 fixed: bash/github/init-yq-workflow.sh:
yq_restore_edited_uniform_diffinaccurate blank lines position
- 🆕 new: ✨ bash/github: added
init-diff-workflow.shto supportdiffcommand workflow - 🆕 new: ✨ bash/github/init-yq-workflow.sh:
yq_restore_edited_uniform_difffunction to restore yaml comments and blank lines - 📝 changed: bash/cache/accum-content.sh: use
yq_restore_edited_uniform_difffunction by default
- 🆕 new: ✨ bash/github/accum-stats.sh: added
STATS_REMOVED_DATESreturn variable, print last removed dates additionally to the last changed dates - 🆕 new: ✨ bash/github/accum-stats.sh:
ENABLE_COMMIT_MESSAGE_DATE_TIME_WITH_LAST_CHANGED_DATE_OFFSETenvironment variable to insert datetime suffix as offset to the last changed date in format-DDTto note the closest changed date
- 🔧 fixed: bash: inpage/accum-downloads.sh, boards/accum-stats.sh: accidental exit with 0 on curl error
- 🔧 fixed: bash: inpage/accum-downloads.sh, boards/accum-stats.sh: incorrect save into
by_yearjson file relativedownloadscounter - 📝 changed: bash: inpage/accum-downloads.sh, boards/accum-stats.sh: removed previous days increment/decrement counters, removed
STATS_PREV_DAY_*return variables
- 🔧 fixed: bash/github/accum-stats.sh:
by_yearfile rewrite on empty changes - 📝 changed: bash/github/accum-stats.sh: made
uniquescounter goes first in all messages
- 🔧 fixed: bash/github/accum-stats.sh: count/uniques minimum update fix
- 🆕 new: ✨ bash/github/accum-stats.sh: added
STATS_CHANGED_DATESreturn variable to indicate the list of changed dates at the current date - 📝 changed: bash/github/accum-stats.sh:
STATS_CURRENT_DATE_*return variables replaced bySTATS_LAST_CHANGED_DATE_*return variable because last changed date and current date are different dates and the last changed date taken into account at the current date - 📝 changed: bash/github/accum-stats.sh: commit message is changed to indicate accumulation for the last changed date only instead of accumulation for all changed dates because the commit message will be placed for all files including
by_year/YYYY/YYYY-MM-DD.jsonfiles
- 🆕 new: ✨ bash/github/accum-stats.sh: added
STATS_CURRENT_DATE_*_INCandSTATS_CURRENT_DATE_*_DECreturn variables - 📝 changed: bash/github/accum-stats.sh: removed previous days increment/decrement counters and replaced by increment/decrement counters in a last date, removed
STATS_PREV_DAY_*return variables
- 🔧 fixed: bash/github/accum-stats.sh: minor changelog output fix
- 🔧 fixed: bash/github/accum-stats.sh: current date count/uniques print into changelog file
- 🆕 new: ✨ bash/github/accum-stats.sh: save changed dates list into changelog file
- 🆕 new: ✨ bash/cache/accum-content.sh: added check of
NO_SKIP_UNEXPIRED_ENTRIESto avoid skip of not yet expired files and force to download them
- 🔧 fixed: bash/inpage/accum-downloads.sh: save into
by_yearjson file relativedownloadscounter instead of absolute - 🔧 fixed: bash/boards/accum-stats.sh: save into
by_yearjson file relativedownloadscounter instead of absolute - 🆕 new: ✨ bash: added
commit_msg_entityoptional input parameter as replacement ofstat_entityin the commit message - 🆕 new: ✨ bash/github/accum-rate-limits.sh: added
stat_entityrequired input parameter to select implementation - 🆕 new: ✨ bash/github/accum-stats.sh: added
STATS_CURRENT_DATE_COUNT,STATS_CURRENT_DATE_UNIQUES,STATS_DATED_COUNT,STATS_DATED_UNIQUES,STATS_OUTDATED_COUNT,STATS_OUTDATED_UNIQUESreturn variables - 📝 changed: bash/github/accum-stats.sh: removed
stats_list_keyinput parameter, usestat_entityto select the implementation - 📝 changed: bash/board/accum-stats.sh: removed
board_nameinput parameter - 📝 changed: bash/board/accum-stats.sh: changed the commit message, replaced previous day increment group by current date accumulated value
- 📝 changed: bash/cache/accum-content.sh: removed
store_entity_pathinput parameter - 📝 changed: bash/cache/accum-content.sh: changed the commit message, replaced previous day increment/decrement group by current date accumulated value
- 📝 changed: bash/inpage/accum-downloads.sh: removed
stat_entity_pathinput parameter - 📝 changed: bash/inpage/accum-downloads.sh: changed the commit message, replaced previous day increment group by current date accumulated value
- 📝 changed: bash: swapped
COMMIT_MESSAGE_PREFIXandCOMMIT_MESSAGE_SUFFIXin all scripts to further improve readability in case of GitHub commit message truncation - 📝 changed: bash: added more accumulation details in the
COMMIT_MESSAGE_PREFIXvalue - 🔀 refactor: bash: renamed
stat_entity_pathtostat_entityin the rest scripts
- 📝 changed: bash/github/accum-content.sh: changed the commit message to improve readability in case of GitHub commit message truncation
- 🔧 fixed: bash: board/accum-stats.sh, cache/accum-content.sh, inpage/accum-downloads.sh: curl progress obscure by stderr is workarounded by grep it into stdout
- 🆕 new: ✨ bash/github/accum-content.sh: generate
content-index.ymlfromcontent-config.ymlif does not exist
- 🔧 fixed: bash/github/accum-content.sh: update downloaded file only if content is changed, increment skip counter if file hash is not changed (additionally to not yet expired files)
- 🆕 new: ✨ bash/github/accum-content.sh:
ERROR_ON_EMPTY_CHANGES_WITHOUT_ERRORSenvironment variable to generate error on empty changes without errors to interrupt futher pipeline execution
- 🔧 fixed: bash/github/accum-content.sh: execution fix
- 🔧 fixed: bash/github/print-*.sh: accidental buffer unset in case of call twice to
gh_enable_print_*_buffering - 🆕 new: ✨ bash: curl stderr print on error
- 🆕 new: ✨ action.yml: added
COMMIT_MESSAGE_PREFIXto split an automated user commit message into prefix and suffix parts - 📝 changed: bash/cache/accum-content.sh: treat empty downloaded files as invalid
- 📝 changed: bash/github/accum-rate-limits.sh: replaced
corestatistic bygraphqlstatistic becauserateduplicates thecorestatistic, addedgraphqlstatistic difference
- 🆕 new: ✨ bash: accum-*.sh: added
ENABLE_COMMIT_MESSAGE_DATE_WITH_TIMEenvironment variable to add date with time to a generated commit message - 📝 changed: bash/github/accum-rate-limits.sh: removed
commit_message_insert_timeas a script parameter - 📝 changed: bash/github/print-*.sh: use script call as call to the
gh_print_*sfunction instead of call to thegh_print_*_lnfunction to print each argument as a single line
- 🆕 new: ✨ bash/github:
init-curl-workflow.shinitialization script - 🆕 new: ✨ bash: print curl response on error support through
ENABLE_PRINT_CURL_RESPONSE_ON_ERRORvariable
- 📝 changed: bash/github/print-*.sh: split
gh_print_*_lnfunctions intogh_print_*_lnandgh_print_*sfunctions to print either all as a single line or all as multi line
- 🔧 fixed: bash/github/init-basic-workflow.sh: split a print buffer by line returns to print
- 🔧 fixed: bash: accum-*.sh: scripts execution
- 🆕 new: ✨ bash: accum-*.sh: enable warnings and errors print buffering on demand to resort warning prints after notices and error prints after warnings and notices
- 🆕 new: ✨ bash: accum-*.sh: added warnings and errors print lag
- 🔧 fixed: bash: accum-*.sh: missed to call
gh_prepend_changelog_fileon early exit
- 🔧 fixed: bash/github/init-basic-workflow.sh: avoid strings excessive expansion in
prepend_changelog_filefunction - 🆕 new: ✨ bash/github:
init-jq-workflow.shinitialization script - 🆕 new: ✨ _externals: partially added as an external copy of the tacklelib bash library
- 📝 changed: bash: accum-*.sh: reduce text output size into the changelog file
- 🔧 fixed: bash: scripts execution
- 🆕 new: ✨ bash/github:
init-basic-workflow.sh,init-stats-workflow.shinitialization scripts - 🆕 new: ✨ bash/github:
set-env-from-args.shscript to set environment variables from arguments - 🆕 new: ✨ bash: changelog file support through
CONTINUE_ON_INVALID_INPUT,CONTINUE_ON_EMPTY_CHANGES,CONTINUE_ON_RESIDUAL_CHANGES,ENABLE_GENERATE_CHANGELOG_FILE,changelog_dir,changelog_txtvariables
- 🆕 new: ✨ bash/github/accum-rate-limit.sh: added
commit_message_insert_timeinput parameter andCOMMIT_MESSAGE_DATE_TIME_PREFIXoutput variable to use in a commit message
- 🆕 new: ✨ bash/github/accum-rate-limit.sh: accumulate rate limits bash script
- 🔧 fixed: bash/github/accum-stats.sh: missed to save by_year json using the current timestamp instead of github json timestamp
- 🔧 fixed: bash//accum-.sh: basic protection from invalid values spread after read an invalid json file
- 🔧 fixed: bash/github/accum-stats.sh: code cleanup
- 🔧 fixed: bash: board/accum-stats.sh, inpage/accum-downloads.sh: exit with error if nothing is changed
- 🔧 fixed: bash/github/accum-stats.sh: missed to check by_year json changes and if changed then change the timestamp
- 🆕 new: ✨ bash: board/accum-stats.sh, inpage/accum-downloads.sh:
STATS_PREV_EXEC_*andSTATS_PREV_DAY_*variables to represent the script previous execution/day difference - 📝 changed: bash: board/accum-stats.sh, inpage/accum-downloads.sh: calculate difference in the
COMMIT_MESSAGE_SUFFIXvariable between current json file and the last change from the previous day instead of from the previous json file (independently to the pipeline scheduler times)
- 🆕 new: ✨ bash//accum-.sh:
STATS_DATE_UTCandSTATS_DATE_TIME_UTCvariables to represent the script execution times - 🆕 new: ✨ bash/github/accum-stats.sh:
STATS_PREV_EXEC_*andSTATS_PREV_DAY_*variables to represent the script previous execution/day difference - 📝 changed: bash/github/accum-stats.sh: calculate difference in the
COMMIT_MESSAGE_SUFFIXvariable between current json file and the last change from the previous day instead of from the previous json file (independently to the pipeline scheduler times)
- 🔧 fixed: bash: missed
stats_dirandstats_jsonvariables check in respective scripts
- 🆕 new: ✨ bash: use
GH_WORKFLOW_ROOTvariable to includegh-workflowshell scripts as dependencies
- 📝 changed: bash/github/print-*.sh: avoid output duplication, always print warnings/errors into stderr including GitHub pipeline
- 🆕 new: ✨ bash/github:
print-*.shscripts to directly call from GitHub pipeline for multiline messages (line per annotation)
- 🆕 new: ✨ accum-stats.sh, accum-downloads.sh: create
COMMIT_MESSAGE_SUFFIXvariable and print statistics change into GitHub pipeline
- 📝 changed: accum-stats.sh, accum-downloads.sh: always print script main execution parameters, even if has no error or warning