Add Democracy Club elections adapter and UK elections pilot - #9
Merged
Merged
Conversation
Democracy Club publishes every UK election - some 42,000 ballots - with voting systems, seats contested, candidacies, parties and declared results. The UK runs several electoral systems side by side, which makes it the right test of the profile's claim that electoralSystem is what lets election data travel. 2 May 2024: 5 elections, 40 contests, 292 candidates, 18 parties. Real results validated first time with no schema change. AMS, the Additional Member System used for the London Assembly, maps to mmp and sits alongside first-past-the-post contests within the same election. With this, every profile has been tested against a real publisher. A latent bug in six SHACL shapes. They listed the jurisdiction class as AdministrativeArea or City only, omitting Country and State, which the _core jurisdiction schema has permitted since it was written. It surfaced here because this is the first election pilot with a national jurisdiction, but it would have failed for any national-level publisher using Post.representsJurisdiction, ServiceRequest.jurisdiction, AlertArea.jurisdiction, Permit.validIn or Contest.representsJurisdiction. Normalised to the full set across all six. A design decision nearly reversed for the wrong reason. The profile deliberately does not store turnout, on the grounds that a percentage should be recomputable from the register. Reading the first sample I concluded Democracy Club publishes turnout_percentage without an electorate, which would have made the headline number of an election unrepresentable, and added a reportedTurnoutPercentage field to carry it. That was wrong: total_electorate is published and I had misread my own probe output. The field was reverted. The original design is visibly better. Barnet and Camden recomputes to 39.5% from 163,655 ballots against an electorate of 413,809, where Democracy Club publishes 40.0% - it rounds turnout to one decimal place, so storing the register recovers the real figure while storing the percentage would have locked in the rounding. That is the argument the profile made, now demonstrated rather than asserted. registeredVoters was added to Contest as well as Election, because the register is drawn per constituency and turnout is only meaningful against the electorate that could vote in that race. Seventeen of forty contests publish no electorate at all. The profile declines to store the derived percentage, so turnout is absent for those rather than recorded at a precision that cannot be recomputed - it prefers a gap to a number it cannot vouch for. Eleven datasets now run in the check pipeline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Democracy Club publishes every UK election - some 42,000 ballots - with voting systems, seats contested, candidacies, parties and declared results. The UK runs several electoral systems side by side, which makes it the right test of the profile's claim that electoralSystem is what lets election data travel.
2 May 2024: 5 elections, 40 contests, 292 candidates, 18 parties. Real results validated first time with no schema change. AMS, the Additional Member System used for the London Assembly, maps to mmp and sits alongside first-past-the-post contests within the same election.
With this, every profile has been tested against a real publisher.
A latent bug in six SHACL shapes. They listed the jurisdiction class as AdministrativeArea or City only, omitting Country and State, which the _core jurisdiction schema has permitted since it was written. It surfaced here because this is the first election pilot with a national jurisdiction, but it would have failed for any national-level publisher using Post.representsJurisdiction, ServiceRequest.jurisdiction, AlertArea.jurisdiction, Permit.validIn or Contest.representsJurisdiction. Normalised to the full set across all six.
A design decision nearly reversed for the wrong reason. The profile deliberately does not store turnout, on the grounds that a percentage should be recomputable from the register. Reading the first sample I concluded Democracy Club publishes turnout_percentage without an electorate, which would have made the headline number of an election unrepresentable, and added a reportedTurnoutPercentage field to carry it. That was wrong: total_electorate is published and I had misread my own probe output. The field was reverted.
The original design is visibly better. Barnet and Camden recomputes to 39.5% from 163,655 ballots against an electorate of 413,809, where Democracy Club publishes 40.0% - it rounds turnout to one decimal place, so storing the register recovers the real figure while storing the percentage would have locked in the rounding. That is the argument the profile made, now demonstrated rather than asserted.
registeredVoters was added to Contest as well as Election, because the register is drawn per constituency and turnout is only meaningful against the electorate that could vote in that race.
Seventeen of forty contests publish no electorate at all. The profile declines to store the derived percentage, so turnout is absent for those rather than recorded at a precision that cannot be recomputed - it prefers a gap to a number it cannot vouch for.
Eleven datasets now run in the check pipeline.