Skip to content

Commit 364b219

Browse files
committed
Add Ancestry free hints automation and tree view fixes
- Ancestry hints: Playwright automation to process free record hints - New "Hints" button on Ancestry row in Provider Data table - Reviews, accepts, and saves hints to tree automatically - SSE progress events, cancellation support, consecutive failure safeguard - Tree views: Simplify connector lines in GenerationalColumnsView - FamilySearch upload: Improve navigation and waiting logic
1 parent 889b720 commit 364b219

14 files changed

Lines changed: 1034 additions & 140 deletions

File tree

‎.changelog/v0.6.x.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,14 @@ This release normalizes FamilySearch as a downstream provider (equal to Ancestry
88

99
## New Features
1010

11+
### Ancestry Free Hints Automation (v0.6.x)
12+
- New "Hints" button on Ancestry row in Provider Data table to process free record hints
13+
- Playwright automation navigates to hints page, reviews each hint, accepts and saves to tree
14+
- Supports checkbox selection for adding related people from hints
15+
- Includes consecutive failure safeguard (stops after 3 failures) and remaining hints re-check
16+
- SSE-based progress events for real-time UI updates
17+
- API endpoints: POST/GET `/api/ancestry-hints/:dbId/:personId`, cancel, status
18+
1119
### Ancestry Vital Info Upload (v0.6.x)
1220
- Extended Ancestry upload to support birth date, birth place, death date, death place, and name (not just photos)
1321
- Upload dialog now shows field-by-field comparison between local SparseTree data and cached Ancestry data

‎PLAN.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ High-level project roadmap. For detailed phase documentation, see [docs/roadmap.
3737
| 15.18 | Separate provider download from auto-apply | ✅ |
3838
| 15.19 | Normalize FamilySearch as downstream provider | ✅ |
3939
| 15.20 | Relationship linking (parents, spouses, children) | 📋 |
40+
| 15.22 | Ancestry free hints automation | ✅ |
4041
| 16 | Multi-platform sync architecture | 📋 |
4142
| 17 | Real-time event system (Socket.IO) | 📋 |
4243

@@ -518,6 +519,48 @@ Enhanced tree views matching Ancestry.com visualization modes:
518519
- `client/src/components/ancestry-tree/AncestryTreeView.tsx` - View switcher dropdown, URL routing
519520
- `client/src/index.css` - Lineage color CSS variables
520521

522+
### Phase 15.22: Ancestry Free Hints Automation ✅
523+
524+
Automates the processing of free hints on Ancestry.com for accepting record hints:
525+
526+
**Features:**
527+
- **Single Person Processing**: Process all free hints for a person via the "Hints" button
528+
- **Browser Automation**: Playwright-based automation that:
529+
- Navigates to hints page with free hints filter
530+
- Clicks "Review" on each hint card
531+
- Accepts the hint by clicking "Yes" to save
532+
- Checks "Add" checkboxes for related people
533+
- Clicks "Save to tree" to complete
534+
- **Progress Tracking**: SSE-based progress events for real-time UI updates
535+
- **Cancellation Support**: In-memory cancellation via Set pattern
536+
- **Rate Limiting**: Uses provider rate limit defaults between hints
537+
538+
**Architecture:**
539+
```
540+
UI Button (ProviderDataTable)
541+
→ API Route (ancestry-hints.routes.ts)
542+
→ Hints Service (ancestry-hints.service.ts)
543+
→ Browser Service (existing Playwright CDP)
544+
→ SSE Progress Events → UI Toast/Progress
545+
```
546+
547+
**Files Created:**
548+
- `server/src/services/ancestry-hints.service.ts` - Core Playwright automation
549+
- `server/src/routes/ancestry-hints.routes.ts` - API endpoints
550+
551+
**Files Modified:**
552+
- `shared/src/index.ts` - `AncestryHintProgress`, `AncestryHintResult` types
553+
- `server/src/index.ts` - Route registration
554+
- `client/src/services/api.ts` - `processAncestryHints()`, `getAncestryHintsStatus()`, `cancelAncestryHints()` methods
555+
- `client/src/components/person/ProviderDataTable.tsx` - "Hints" button with Zap icon
556+
- `client/src/components/person/PersonDetail.tsx` - Handler and state
557+
558+
**API Endpoints:**
559+
- `POST /api/ancestry-hints/:dbId/:personId` - Process hints for a person
560+
- `GET /api/ancestry-hints/:dbId/:personId/events` - SSE progress stream
561+
- `POST /api/ancestry-hints/:dbId/cancel` - Cancel running operation
562+
- `GET /api/ancestry-hints/status` - Check if running
563+
521564
### Phase 16: Multi-Platform Sync (Remaining Items)
522565

523566
- ~~Provider cache structure~~ (completed in 15.11)

‎client/src/components/ancestry-tree/AncestryTreeView.tsx‎

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -286,16 +286,8 @@ export function AncestryTreeView() {
286286
<GenerationalColumnsView
287287
data={treeData}
288288
dbId={dbId!}
289-
onLoadMore={async (newDepth: number) => {
290-
if (!dbId || !rootId) return;
291-
setLoading(true);
292-
const data = await api.getAncestryTree(dbId, rootId, newDepth).catch(err => {
293-
console.error('Failed to load more generations:', err);
294-
return null;
295-
});
296-
if (data) setTreeData(data);
297-
setLoading(false);
298-
}}
289+
onExpand={handleExpand}
290+
expandingNodes={expandingNodes}
299291
/>
300292
)}
301293

‎client/src/components/ancestry-tree/views/GenerationalColumnsView.tsx‎

Lines changed: 42 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,17 @@
44
* Displays ancestors in vertical columns organized by generation.
55
* Root person on the left, with each generation flowing to the right.
66
* Scrollable horizontally for deeper ancestry.
7+
* Users can expand individual nodes to load more ancestors.
78
*/
89
import { useEffect, useState } from 'react';
9-
import { Link } from 'react-router-dom';
10-
import type { AncestryTreeResult, AncestryPersonCard, AncestryFamilyUnit } from '@fsf/shared';
11-
import { AvatarPlaceholder } from '../../avatars/AvatarPlaceholder';
10+
import type { AncestryTreeResult, AncestryPersonCard, AncestryFamilyUnit, ExpandAncestryRequest } from '@fsf/shared';
11+
import { AncestorNode, RootPersonNode } from '../shared/AncestorNode';
1212

1313
interface GenerationalColumnsViewProps {
1414
data: AncestryTreeResult;
1515
dbId: string;
16-
onLoadMore?: (newDepth: number) => Promise<void>;
16+
onExpand?: (request: ExpandAncestryRequest, nodeId: string) => void;
17+
expandingNodes?: Set<string>;
1718
}
1819

1920
interface GenerationPerson {
@@ -76,56 +77,6 @@ function buildGenerations(data: AncestryTreeResult, maxGen: number): Generation[
7677
return generations;
7778
}
7879

79-
interface PersonCardProps {
80-
person: AncestryPersonCard;
81-
dbId: string;
82-
compact?: boolean;
83-
}
84-
85-
function PersonCard({ person, dbId, compact = false }: PersonCardProps) {
86-
const isMale = person.gender === 'male';
87-
const gender = person.gender === 'female' ? 'female' : person.gender === 'male' ? 'male' : 'unknown';
88-
89-
if (compact) {
90-
return (
91-
<Link
92-
to={`/person/${dbId}/${person.id}`}
93-
className={`flex items-center gap-2 p-2 rounded-lg border-l-4 ${isMale ? 'border-l-app-male bg-app-male/10 hover:bg-app-male/15' : 'border-l-app-female bg-app-female/10 hover:bg-app-female/15'} transition-colors min-w-[160px]`}
94-
>
95-
<div className={`w-8 h-8 rounded-full border-2 ${isMale ? 'border-app-male' : 'border-app-female'} flex items-center justify-center flex-shrink-0 overflow-hidden`}>
96-
{person.photoUrl ? (
97-
<img src={person.photoUrl} alt="" className="w-full h-full object-cover" />
98-
) : (
99-
<AvatarPlaceholder gender={gender} className="w-full h-full" />
100-
)}
101-
</div>
102-
<div className="min-w-0 flex-1">
103-
<div className="font-medium text-app-text text-xs truncate">{person.name}</div>
104-
<div className="text-[10px] text-app-text-muted">{person.lifespan}</div>
105-
</div>
106-
</Link>
107-
);
108-
}
109-
110-
return (
111-
<Link
112-
to={`/person/${dbId}/${person.id}`}
113-
className={`flex items-center gap-3 p-3 rounded-xl border-l-4 ${isMale ? 'border-l-app-male bg-app-male/10 hover:bg-app-male/20' : 'border-l-app-female bg-app-female/10 hover:bg-app-female/20'} transition-colors min-w-[200px]`}
114-
>
115-
<div className={`w-12 h-12 rounded-full border-2 ${isMale ? 'border-app-male' : 'border-app-female'} flex items-center justify-center flex-shrink-0 overflow-hidden`}>
116-
{person.photoUrl ? (
117-
<img src={person.photoUrl} alt="" className="w-full h-full object-cover" />
118-
) : (
119-
<AvatarPlaceholder gender={gender} className="w-full h-full" />
120-
)}
121-
</div>
122-
<div className="min-w-0 flex-1">
123-
<div className="font-medium text-app-text text-sm">{person.name}</div>
124-
<div className="text-xs text-app-text-muted">{person.lifespan}</div>
125-
</div>
126-
</Link>
127-
);
128-
}
12980

13081
// Get generation label - simplified for deep generations
13182
function getGenerationLabel(level: number): { main: string; sub?: string } {
@@ -144,16 +95,15 @@ function getOrdinalSuffix(n: number): string {
14495
return s[(v - 20) % 10] || s[v] || s[0];
14596
}
14697

147-
export function GenerationalColumnsView({ data, dbId, onLoadMore }: GenerationalColumnsViewProps) {
98+
export function GenerationalColumnsView({ data, dbId, onExpand, expandingNodes = new Set() }: GenerationalColumnsViewProps) {
14899
const [generations, setGenerations] = useState<Generation[]>([]);
149100
const [maxGen, setMaxGen] = useState(data.maxGenerationLoaded);
150-
const [loadingMore, setLoadingMore] = useState(false);
151101

152102
useEffect(() => {
153103
setGenerations(buildGenerations(data, maxGen));
154104
}, [data, maxGen]);
155105

156-
// Update maxGen when data changes (after loading more)
106+
// Update maxGen when data changes (after expanding nodes)
157107
useEffect(() => {
158108
if (data.maxGenerationLoaded > maxGen) {
159109
setMaxGen(data.maxGenerationLoaded);
@@ -163,18 +113,20 @@ export function GenerationalColumnsView({ data, dbId, onLoadMore }: Generational
163113
// Count known ancestors per generation
164114
const getKnownCount = (gen: Generation) => gen.people.filter(p => p !== null).length;
165115

166-
// Check if any person in the last generation has more ancestors
167-
const hasMoreToLoad = (): boolean => {
168-
if (generations.length === 0) return false;
169-
const lastGen = generations[generations.length - 1];
170-
return lastGen.people.some(p => p?.person.hasMoreAncestors);
116+
// Count how many nodes have more ancestors to load
117+
const getExpandableCount = (): number => {
118+
return generations.reduce((sum, gen) => {
119+
return sum + gen.people.filter(p => p?.person.hasMoreAncestors).length;
120+
}, 0);
171121
};
172122

173-
const handleLoadMore = async () => {
174-
if (!onLoadMore || loadingMore) return;
175-
setLoadingMore(true);
176-
await onLoadMore(data.maxGenerationLoaded + 5);
177-
setLoadingMore(false);
123+
// Handle expanding a person's ancestors
124+
const handleNodeExpand = (person: AncestryPersonCard) => {
125+
if (!onExpand) return;
126+
const request: ExpandAncestryRequest = person.gender === 'female'
127+
? { motherId: person.id }
128+
: { fatherId: person.id };
129+
onExpand(request, person.id);
178130
};
179131

180132
return (
@@ -183,7 +135,9 @@ export function GenerationalColumnsView({ data, dbId, onLoadMore }: Generational
183135
<div className="px-4 py-3 bg-app-card border-b border-app-border flex items-center justify-between">
184136
<div className="text-sm text-app-text-muted">
185137
Showing {generations.length} generations ({generations.reduce((sum, g) => sum + getKnownCount(g), 0)} ancestors)
186-
{hasMoreToLoad() && <span className="text-app-text-subtle ml-2">&bull; More available</span>}
138+
{getExpandableCount() > 0 && (
139+
<span className="text-app-text-subtle ml-2">&bull; {getExpandableCount()} expandable</span>
140+
)}
187141
</div>
188142
<div className="flex items-center gap-3">
189143
<span className="text-xs text-app-text-subtle">Visible:</span>
@@ -235,15 +189,27 @@ export function GenerationalColumnsView({ data, dbId, onLoadMore }: Generational
235189
{/* People in this generation - only show known people */}
236190
<div className="flex-1 p-3 flex flex-col">
237191
<div className="flex flex-col gap-1.5">
238-
{knownPeople.map((item, idx) => (
239-
<PersonCard
240-
key={`${gen.level}-${idx}-${item.person.id}`}
241-
person={item.person}
192+
{gen.level === 0 ? (
193+
<RootPersonNode
194+
key={`root-${data.rootPerson.id}`}
195+
person={data.rootPerson}
242196
dbId={dbId}
243-
compact={gen.level > 2}
244197
/>
245-
))}
246-
{knownPeople.length === 0 && (
198+
) : (
199+
knownPeople.map((item, idx) => (
200+
<AncestorNode
201+
key={`${gen.level}-${idx}-${item.person.id}`}
202+
person={item.person}
203+
dbId={dbId}
204+
size={gen.level > 3 ? 'xs' : gen.level > 2 ? 'sm' : 'md'}
205+
variant="card"
206+
expandDirection="right"
207+
onExpand={item.person.hasMoreAncestors && onExpand ? () => handleNodeExpand(item.person) : undefined}
208+
isExpanding={expandingNodes.has(item.person.id)}
209+
/>
210+
))
211+
)}
212+
{gen.level > 0 && knownPeople.length === 0 && (
247213
<div className="text-xs text-app-text-muted p-2 text-center">
248214
No known ancestors
249215
</div>
@@ -253,39 +219,6 @@ export function GenerationalColumnsView({ data, dbId, onLoadMore }: Generational
253219
</div>
254220
);
255221
})}
256-
257-
{/* Load More column */}
258-
{hasMoreToLoad() && onLoadMore && (
259-
<div className="flex flex-col min-w-[140px] border-r-0">
260-
<div className="px-4 py-2 bg-app-card border-b border-app-border sticky top-0 z-10">
261-
<div className="text-xs font-bold text-app-text uppercase tracking-wide">
262-
More...
263-
</div>
264-
<div className="text-[10px] text-app-text-muted">
265-
Load deeper ancestry
266-
</div>
267-
</div>
268-
<div className="flex-1 p-3 flex items-start justify-center">
269-
<button
270-
onClick={handleLoadMore}
271-
disabled={loadingMore}
272-
className="flex flex-col items-center gap-2 p-4 rounded-lg bg-app-border hover:bg-app-hover disabled:opacity-50 disabled:cursor-wait transition-colors"
273-
>
274-
{loadingMore ? (
275-
<>
276-
<div className="w-6 h-6 border-2 border-app-text-muted border-t-transparent rounded-full animate-spin" />
277-
<span className="text-xs text-app-text-muted">Loading...</span>
278-
</>
279-
) : (
280-
<>
281-
<span className="text-2xl">+</span>
282-
<span className="text-xs text-app-text-muted">Load 5 more</span>
283-
</>
284-
)}
285-
</button>
286-
</div>
287-
</div>
288-
)}
289222
</div>
290223
</div>
291224

@@ -297,7 +230,7 @@ export function GenerationalColumnsView({ data, dbId, onLoadMore }: Generational
297230
<span className="flex items-center gap-1">
298231
<span className="w-3 h-1 bg-app-female"></span> Female
299232
</span>
300-
<span>Scroll horizontally to see more generations</span>
233+
<span>Click arrows on nodes to expand ancestors</span>
301234
</div>
302235
</div>
303236
);

‎client/src/components/ancestry-tree/views/VerticalFamilyView.tsx‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -527,9 +527,6 @@ export function VerticalFamilyView({
527527
const nodeMap = new Map<string, PositionedNode>();
528528
nodes.forEach(n => nodeMap.set(n.id, n));
529529

530-
// Find max generation to calculate track offsets
531-
const maxGen = Math.max(...nodes.map(n => n.generation), 0);
532-
533530
// Group nodes with parents by generation to assign sub-tracks
534531
const nodesWithParentsByGen = new Map<number, PositionedNode[]>();
535532
nodes.forEach(node => {

‎client/src/components/person/ProviderDataTable.tsx‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { useState, useEffect, useRef } from 'react';
2-
import { ExternalLink, Download, Upload, Camera, Link2, Loader2, Check, AlertCircle, User, ArrowRight, ChevronDown } from 'lucide-react';
2+
import { ExternalLink, Download, Upload, Camera, Link2, Loader2, Check, AlertCircle, User, ArrowRight, ChevronDown, Zap } from 'lucide-react';
33
import toast from 'react-hot-toast';
44
import type { PersonAugmentation, MultiPlatformComparison, BuiltInProvider, ComparisonStatus, PlatformType } from '@fsf/shared';
55
import { api } from '../../services/api';
@@ -37,9 +37,11 @@ interface ProviderDataTableProps {
3737
onShowLinkInput: (platform: 'wikipedia' | 'ancestry' | 'wikitree' | 'linkedin') => void;
3838
onPhotoChanged?: () => void; // Called when primary photo changes to refresh parent state
3939
onFieldChanged?: () => void; // Called when a field value is applied to refresh parent state
40+
onProcessAncestryHints?: () => Promise<void>; // Called to process free hints on Ancestry
4041
syncLoading: boolean;
4142
scrapeLoading: boolean;
4243
fetchingPhotoFrom: string | null;
44+
hintsProcessing?: boolean; // True when hints are being processed
4345
}
4446

4547
// Provider display info
@@ -594,9 +596,11 @@ export function ProviderDataTable({
594596
onShowLinkInput,
595597
onPhotoChanged,
596598
onFieldChanged,
599+
onProcessAncestryHints,
597600
syncLoading,
598601
scrapeLoading,
599602
fetchingPhotoFrom,
603+
hintsProcessing,
600604
}: ProviderDataTableProps) {
601605
const [comparison, setComparison] = useState<MultiPlatformComparison | null>(null);
602606
const [refreshingProvider, setRefreshingProvider] = useState<string | null>(null);
@@ -1004,6 +1008,17 @@ export function ProviderDataTable({
10041008
>
10051009
<Upload size={10} />
10061010
</button>
1011+
{onProcessAncestryHints && (
1012+
<button
1013+
onClick={onProcessAncestryHints}
1014+
disabled={hintsProcessing}
1015+
className={`inline-flex items-center gap-1 px-1.5 py-0.5 rounded text-xs ${PROVIDER_INFO.ancestry.bgColor} ${PROVIDER_INFO.ancestry.color} hover:opacity-80 disabled:opacity-50`}
1016+
title="Process free hints on Ancestry"
1017+
>
1018+
{hintsProcessing ? <Loader2 size={10} className="animate-spin" /> : <Zap size={10} />}
1019+
Hints
1020+
</button>
1021+
)}
10071022
</>
10081023
) : (
10091024
<button

0 commit comments

Comments
 (0)