feat: review all remote HolodoriDB changes before collection
This commit is contained in:
@@ -15,3 +15,16 @@
|
||||
- Verify repository changes with the relevant tests, `npm run check`, `npm run build`, and `npm run build:pages` when preparing a deployment.
|
||||
- Do not bump the package version, create version-specific release notes, create tags, or publish GitHub Releases for routine online commits and deployments unless the user explicitly requests them.
|
||||
- Use `UPDATES.md` as the single user-facing update history and include only changes that affect service users.
|
||||
|
||||
## HolodoriDB source policy
|
||||
|
||||
- Always fetch the official HolodoriDB remote main before collecting or checking game data and strings. Read files at the fetched commit, not local checkout files or stale remote-tracking refs.
|
||||
- Use scripts/holodoridb-source.mjs for collectors. A failed remote fetch must stop collection; do not fall back to cached data or --offline.
|
||||
- Resolve Korean song titles through Music.json titleLangId and LangMusic_Kor.json before asking the user for names.
|
||||
|
||||
## Collection review before calculator extraction
|
||||
|
||||
- A collection request starts with `npm run data:sync`: compare all three official remotes against the last successful collection commits in data/holodoridb/manifest.json. Inspect every intervening commit and every changed file, including all strings, tables unrelated to the calculator, and changes later reverted.
|
||||
- Read data/holodoridb-review/latest.json (full JSON before/after details), latest.md, and changes.patch. Present a Korean semantic summary grouped by content (cards/skills, songs, events, translations, other systems), describing additions, removals, corrections, and relevant dates. Do not substitute a file list or calculator-only summary for this review; state unknown meanings explicitly.
|
||||
- Provide this report to the user before acquiring/extracting calculator data. Then continue with `npm run data:apply` and other required collectors within the authorized task; no additional approval is implied. A review-only request must not extract data.
|
||||
- Reviewing does not advance the collection baseline. Extraction must use the reviewed remote commits; if any remote changes, regenerate and present the report first.
|
||||
|
||||
@@ -177,13 +177,15 @@ docker compose -f compose.prod.yaml up --build -d
|
||||
<details>
|
||||
<summary><strong>게임 데이터 갱신과 카드 표시명 수정</strong></summary>
|
||||
|
||||
공개 HolodoriDB diff 저장소에서 카드 관련 마스터 데이터와 한국어·영어·일본어 텍스트를 수집합니다.
|
||||
먼저 공개 HolodoriDB 세 언어 원격 저장소에서 마지막 수집 이후의 모든 커밋·파일·문자열 변경을 검토합니다. 계산기와 관계없는 변경도 포함하며, 중간에 되돌려진 변경도 커밋별로 기록합니다.
|
||||
|
||||
```bash
|
||||
npm run data:sync
|
||||
```
|
||||
|
||||
생성된 데이터는 `data/holodoridb`에 저장합니다. 카드 일러스트, 영상과 음성은 수집 대상에서 제외합니다.
|
||||
검토 보고서는 `data/holodoridb-review/latest.md`, JSON 변경 전후 값은 `latest.json`, 전체 원문 변경은 `changes.patch`에 저장합니다. 보고서를 읽고 변경의 의미를 사용자에게 먼저 제공한 뒤 `npm run data:apply`로 계산기 데이터를 수집합니다. 검토만으로 마지막 수집 기준이 갱신되지는 않습니다.
|
||||
|
||||
생성된 계산기 데이터는 `data/holodoridb`에 저장합니다. 카드 일러스트, 영상과 음성은 수집 대상에서 제외합니다.
|
||||
|
||||
특별 버전명이나 번역 이름은 `data/card-overrides.csv`에서 관리합니다.
|
||||
|
||||
@@ -196,6 +198,6 @@ card-06002-5-uniq-0066-00,수영복,
|
||||
- `versionName`: 이름 옆에 표시할 짧은 버전명
|
||||
- `displayName`: 기본 멤버 이름을 완전히 바꿀 때만 입력
|
||||
|
||||
수정 후 `npm run data:sync`를 실행하면 반영됩니다. 이 파일은 원본 데이터를 다시 수집해도 덮어쓰지 않습니다.
|
||||
수정 후 `npm run data:sync`로 전체 변경을 검토·보고한 다음 `npm run data:apply`를 실행하면 반영됩니다. 이 파일은 원본 데이터를 다시 수집해도 덮어쓰지 않습니다.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# HolodoriDB card data
|
||||
|
||||
This directory is generated by `npm run data:sync`.
|
||||
First run `npm run data:sync` to review every remote change since the commits in manifest.json. The review includes every intervening commit, all files, JSON record/field before-and-after values, and full patches under data/holodoridb-review. Present a semantic summary to the user before calculator extraction. Review does not advance the collection baseline.
|
||||
|
||||
This directory is then generated by `npm run data:apply`. Collectors reject remote commits that do not match the review report.
|
||||
|
||||
- `manifest.json`: source commits, master-data version, checksums, and validation counts
|
||||
- `card-master.json`: lossless copies of the card-related master tables and Korean/English/Japanese localization tables
|
||||
@@ -9,6 +11,6 @@ This directory is generated by `npm run data:sync`.
|
||||
|
||||
The collection script also creates `src/lib/data/cards-summary.json`, a small Korean-only projection used by the card list page.
|
||||
|
||||
The ignored source clones are cached under `data/raw/holodoridb` so a later sync only needs to fetch changes.
|
||||
All HolodoriDB collectors fetch the official remote `main` before reading data. A bare object cache under `data/raw/holodoridb-remote` only accelerates fetching; reads are pinned to the fetched commit, never checkout files. Remote failures stop collection, and `--offline` is rejected. Song metadata and titles come from `Music.json` and `LangMusic_Kor.json` in the Korean remote repository.
|
||||
|
||||
Binary assets such as card illustrations, movies, and voices are not collected. The source repositories did not contain a license file at the recorded commits, so redistribution terms must be confirmed before publishing the generated data as part of a public service.
|
||||
|
||||
+2
-1
@@ -17,7 +17,8 @@
|
||||
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
|
||||
"test:engine": "node --test src/lib/engine/*.test.ts src/lib/analytics/*.test.ts",
|
||||
"start": "node build",
|
||||
"data:sync": "node scripts/collect-holodoridb.mjs",
|
||||
"data:sync": "node scripts/review-holodoridb.mjs",
|
||||
"data:apply": "node scripts/collect-holodoridb.mjs",
|
||||
"data:import-songs": "node scripts/import-legacy-song-data.mjs",
|
||||
"data:collect-charts": "node scripts/collect-exact-charts.mjs",
|
||||
"data:validate-score-calibrations": "node scripts/validate-score-calibrations.mjs",
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { readHolodoriDbTable as readHolodoriDbSource } from './holodoridb-source.mjs';
|
||||
import path from 'node:path';
|
||||
import { formulaScoreMetadata } from './song-score-formulas.mjs';
|
||||
|
||||
@@ -13,19 +13,10 @@ const scoreCalibrationByKey = new Map(scoreCalibrations.charts.map((row) => [
|
||||
`${row.songId}::${row.difficulty}`,
|
||||
row
|
||||
]));
|
||||
function readHolodoriDbSource(locale, filename) {
|
||||
const repositoryPath = path.resolve(`data/raw/holodoridb/${locale}`);
|
||||
return JSON.parse(execFileSync('git', [
|
||||
'-c', `safe.directory=${repositoryPath.replaceAll('\\', '/')}`,
|
||||
'-C', repositoryPath,
|
||||
'show', `origin/main:${filename}`
|
||||
], { encoding: 'utf8', maxBuffer: 32 * 1024 * 1024 }));
|
||||
}
|
||||
|
||||
const musicMaster = readHolodoriDbSource('kor', 'Music.json');
|
||||
const musicById = new Map(musicMaster.map((row) => [row.data?.id ?? row.id, row.data ?? row]));
|
||||
const englishMusicText = readHolodoriDbSource('eng', 'LangMusic_Eng.json');
|
||||
const englishTextById = new Map(englishMusicText.map((row) => [row.data?.id ?? row.id, row.data?.text ?? row.text]));
|
||||
const koreanMusicText = readHolodoriDbSource('kor', 'LangMusic_Kor.json');
|
||||
const koreanTextById = new Map(koreanMusicText.map((row) => [row.data?.id ?? row.id, row.data?.text ?? row.text]));
|
||||
const musicDifficultyMaster = readHolodoriDbSource('kor', 'MusicDifficulty.json');
|
||||
const difficultyLevelByKey = new Map(musicDifficultyMaster.map((row) => {
|
||||
const data = row.data ?? row;
|
||||
@@ -266,7 +257,7 @@ async function worker() {
|
||||
await writeFile(path.join(outputDirectory, fileName), JSON.stringify(output), 'utf8');
|
||||
normalizedCharts.push({
|
||||
songId: item.songId,
|
||||
title: englishTextById.get(`la-music_title-${item.songId}`) ?? sourceEntry?.title ?? item.songId,
|
||||
title: koreanTextById.get(musicById.get(item.songId)?.titleLangId ?? `la-music_title-${item.songId}`) ?? item.songId,
|
||||
difficulty: item.difficulty.toLowerCase(),
|
||||
difficultyLevel: difficultyLevelByKey.get(`${item.songId}::${item.difficulty.toLowerCase()}`) ??
|
||||
(Number.isFinite(item.difficultyLevel) ? item.difficultyLevel : null),
|
||||
|
||||
@@ -1,16 +1,14 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { prepareHolodoriDbSource, readHolodoriDbText } from './holodoridb-source.mjs';
|
||||
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import process from 'node:process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const projectRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const rawRoot = path.join(projectRoot, 'data', 'raw', 'holodoridb');
|
||||
const outputRoot = path.join(projectRoot, 'data', 'holodoridb');
|
||||
const appDataRoot = path.join(projectRoot, 'src', 'lib', 'data');
|
||||
const cardOverridesPath = path.join(projectRoot, 'data', 'card-overrides.csv');
|
||||
const offline = process.argv.includes('--offline');
|
||||
|
||||
function parseCsvLine(line) {
|
||||
const values = [];
|
||||
@@ -136,52 +134,12 @@ const localizationTableNames = [
|
||||
'SkillTreePoint'
|
||||
];
|
||||
|
||||
function runGit(args, options = {}) {
|
||||
return execFileSync('git', args, {
|
||||
cwd: projectRoot,
|
||||
encoding: 'utf8',
|
||||
stdio: ['ignore', 'pipe', 'inherit'],
|
||||
...options
|
||||
}).trim();
|
||||
}
|
||||
|
||||
function gitAt(repositoryPath, args) {
|
||||
const normalizedPath = repositoryPath.replaceAll('\\', '/');
|
||||
return runGit(['-c', `safe.directory=${normalizedPath}`, '-C', repositoryPath, ...args]);
|
||||
}
|
||||
|
||||
async function prepareSource(locale, source) {
|
||||
const repositoryPath = path.join(rawRoot, locale);
|
||||
|
||||
try {
|
||||
gitAt(repositoryPath, ['rev-parse', '--git-dir']);
|
||||
if (offline) {
|
||||
process.stdout.write(`Using cached ${locale} source...\n`);
|
||||
} else {
|
||||
process.stdout.write(`Updating ${locale} source...\n`);
|
||||
gitAt(repositoryPath, ['fetch', '--depth', '1', 'origin', 'main']);
|
||||
}
|
||||
} catch {
|
||||
process.stdout.write(`Cloning ${locale} source...\n`);
|
||||
await mkdir(path.dirname(repositoryPath), { recursive: true });
|
||||
runGit(['clone', '--depth', '1', source.repository, repositoryPath]);
|
||||
}
|
||||
|
||||
const commit = gitAt(repositoryPath, ['rev-parse', 'origin/main']);
|
||||
const commitDate = gitAt(repositoryPath, ['show', '-s', '--format=%cI', commit]);
|
||||
const commitMessage = gitAt(repositoryPath, ['show', '-s', '--format=%s', commit]);
|
||||
|
||||
return {
|
||||
...source,
|
||||
repositoryPath,
|
||||
commit,
|
||||
commitDate,
|
||||
commitMessage
|
||||
};
|
||||
function prepareSource(locale, source) {
|
||||
return { ...source, ...prepareHolodoriDbSource(locale) };
|
||||
}
|
||||
|
||||
function readSourceFile(source, filename) {
|
||||
return gitAt(source.repositoryPath, ['show', `${source.commit}:${filename}`]);
|
||||
return readHolodoriDbText(source, filename);
|
||||
}
|
||||
|
||||
function parseSourceTable(source, filename) {
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import { readHolodoriDbTable } from './holodoridb-source.mjs';
|
||||
import { readFile, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
const projectRoot = path.resolve(path.dirname(new URL(import.meta.url).pathname.replace(/^\/(?=[A-Za-z]:)/, '')), '..');
|
||||
const catalogPath = path.join(projectRoot, 'src/lib/data/exact-chart-catalog.json');
|
||||
const musicPath = path.join(projectRoot, 'data/raw/holodoridb/kor/Music.json');
|
||||
const musicRows = JSON.parse(await readFile(musicPath, 'utf8'));
|
||||
const musicRows = readHolodoriDbTable('kor', 'Music.json');
|
||||
const musicById = new Map(musicRows.map((row) => [row.data?.id ?? row.id, row.data ?? row]));
|
||||
const singerMetadataOverrides = {
|
||||
m0334: { singerType: 'solo', singerCharacterIds: ['chr-04001'] }
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
import { isDeepStrictEqual } from 'node:util';
|
||||
|
||||
// Use row IDs so insertion/reordering does not masquerade as changed records.
|
||||
export function jsonChanges(before, after, location = '$') {
|
||||
if (isDeepStrictEqual(before, after)) return [];
|
||||
if (Array.isArray(before) && Array.isArray(after)) {
|
||||
const keyed = rows => rows.every(row => row && typeof row === 'object' && (row.id ?? row.data?.id) != null)
|
||||
&& new Set(rows.map(row => row.id ?? row.data.id)).size === rows.length;
|
||||
if (keyed(before) && keyed(after)) {
|
||||
const index = rows => Object.fromEntries(rows.map(row => [row.id ?? row.data.id, row]));
|
||||
return jsonChanges(index(before), index(after), location);
|
||||
}
|
||||
}
|
||||
if (before && after && typeof before === 'object' && typeof after === 'object'
|
||||
&& !Array.isArray(before) && !Array.isArray(after)) {
|
||||
return [...new Set([...Object.keys(before), ...Object.keys(after)])].flatMap(key =>
|
||||
jsonChanges(before[key], after[key], `${location}[${JSON.stringify(key)}]`));
|
||||
}
|
||||
return [{ path: location, type: before === undefined ? 'added' : after === undefined ? 'deleted' : 'modified',
|
||||
...(before === undefined ? {} : { before }), ...(after === undefined ? {} : { after }) }];
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { jsonChanges } from './holodoridb-diff.mjs';
|
||||
|
||||
test('row reordering is ignored while string edits retain ID and before/after', () => {
|
||||
const before = [{ id: 'a', data: { text: 'old' } }, { id: 'b', data: { text: 'same' } }];
|
||||
const after = [{ id: 'b', data: { text: 'same' } }, { id: 'a', data: { text: 'new' } }];
|
||||
assert.deepEqual(jsonChanges(before, after), [{ path: '$["a"]["data"]["text"]', type: 'modified', before: 'old', after: 'new' }]);
|
||||
});
|
||||
test('added and deleted records retain full values', () => {
|
||||
const changes = jsonChanges([{ id: 'old', value: 2 }], [{ id: 'new', value: 3 }]);
|
||||
assert.equal(changes[0].type, 'deleted');
|
||||
assert.deepEqual(changes[0].before, { id: 'old', value: 2 });
|
||||
assert.equal(changes[1].type, 'added');
|
||||
assert.deepEqual(changes[1].after, { id: 'new', value: 3 });
|
||||
});
|
||||
test('nulls and arrays without IDs are retained without losing changes', () => {
|
||||
assert.deepEqual(jsonChanges({ a: null, b: [1, 2] }, { a: false, b: [2, 1] }).map(c => c.type), ['modified', 'modified']);
|
||||
});
|
||||
@@ -0,0 +1,66 @@
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdirSync, readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const projectRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const prepared = new Map();
|
||||
|
||||
export function holodoriGit(repositoryPath, args) {
|
||||
return execFileSync('git', ['-c', `safe.directory=${repositoryPath.replaceAll('\\', '/')}`, '-C', repositoryPath, ...args], {
|
||||
encoding: 'utf8', maxBuffer: 64 * 1024 * 1024, stdio: ['ignore', 'pipe', 'pipe']
|
||||
}).trim();
|
||||
}
|
||||
const git = holodoriGit;
|
||||
|
||||
// Each process fetches the authoritative remote once per locale, then pins all
|
||||
// reads to that commit. Neither checkout files nor stale remote refs are inputs.
|
||||
export function prepareHolodoriDbSource(locale, { review = false } = {}) {
|
||||
if (!['eng', 'kor', 'jpn'].includes(locale)) throw new Error(`Unknown HolodoriDB locale: ${locale}`);
|
||||
if (process.argv.includes('--offline')) throw new Error('HolodoriDB collection requires the remote repository; --offline is not supported.');
|
||||
if (prepared.has(locale)) {
|
||||
const source = prepared.get(locale);
|
||||
if (!review) requireReviewedSource(locale, source);
|
||||
return source;
|
||||
}
|
||||
const repository = `https://github.com/HolodoriDB/holodori-db-${locale}-diff.git`;
|
||||
const repositoryPath = path.join(projectRoot, 'data', 'raw', 'holodoridb-remote', locale);
|
||||
mkdirSync(repositoryPath, { recursive: true });
|
||||
git(repositoryPath, ['init', '--bare']);
|
||||
try {
|
||||
git(repositoryPath, ['fetch', '--depth', '1', '--no-tags', repository, '+refs/heads/main:refs/heads/source-main']);
|
||||
} catch (error) {
|
||||
throw new Error(`Cannot fetch HolodoriDB ${locale} remote main; collection stopped without using cached data.`, { cause: error });
|
||||
}
|
||||
const commit = git(repositoryPath, ['rev-parse', 'refs/heads/source-main']);
|
||||
const source = {
|
||||
repository, repositoryPath, commit,
|
||||
commitDate: git(repositoryPath, ['show', '-s', '--format=%cI', commit]),
|
||||
commitMessage: git(repositoryPath, ['show', '-s', '--format=%s', commit])
|
||||
};
|
||||
prepared.set(locale, source);
|
||||
if (!review) requireReviewedSource(locale, source);
|
||||
return source;
|
||||
}
|
||||
|
||||
function requireReviewedSource(locale, source) {
|
||||
let report;
|
||||
try {
|
||||
report = JSON.parse(readFileSync(path.join(projectRoot, 'data/holodoridb-review/latest.json'), 'utf8'));
|
||||
} catch {
|
||||
throw new Error('Run npm run data:sync and present the full change report before collecting calculator data.');
|
||||
}
|
||||
const manifest = JSON.parse(readFileSync(path.join(projectRoot, 'data/holodoridb/manifest.json'), 'utf8'));
|
||||
const entry = report.sources?.[locale];
|
||||
if (entry?.to !== source.commit || ![entry.from, entry.to].includes(manifest.sources[locale].commit)) {
|
||||
throw new Error(`HolodoriDB ${locale} changed since review. Run npm run data:sync and present the updated report first.`);
|
||||
}
|
||||
}
|
||||
|
||||
export function readHolodoriDbText(source, filename) {
|
||||
return git(source.repositoryPath, ['show', `${source.commit}:${filename}`]);
|
||||
}
|
||||
|
||||
export function readHolodoriDbTable(locale, filename) {
|
||||
return JSON.parse(readHolodoriDbText(prepareHolodoriDbSource(locale), filename));
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { prepareHolodoriDbSource, holodoriGit as git } from './holodoridb-source.mjs';
|
||||
import { jsonChanges } from './holodoridb-diff.mjs';
|
||||
|
||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const output = path.join(root, 'data/holodoridb-review');
|
||||
const baselineIndex = process.argv.indexOf('--baseline-ref');
|
||||
const baselineRef = baselineIndex < 0 ? null : process.argv[baselineIndex + 1];
|
||||
if (baselineIndex >= 0 && (!baselineRef || baselineRef.startsWith('-'))) throw new Error('--baseline-ref requires a git revision.');
|
||||
const manifest = JSON.parse(baselineRef
|
||||
? git(root, ['show', `${baselineRef}:data/holodoridb/manifest.json`])
|
||||
: await readFile(path.join(root, 'data/holodoridb/manifest.json'), 'utf8'));
|
||||
const report = { generatedAt: new Date().toISOString(), baselineCollectedAt: manifest.generatedAt, sources: {} };
|
||||
const lines = ['# HolodoriDB 전체 변경 검토', '', `마지막 수집: ${manifest.generatedAt}`, `검토 시각: ${report.generatedAt}`, '',
|
||||
'계산기 필요 여부로 필터링하지 않은 전체 원격 변경입니다. 검토는 수집 기준 커밋이나 계산기 데이터를 갱신하지 않습니다.', ''];
|
||||
const patches = [];
|
||||
for (const locale of ['eng', 'kor', 'jpn']) {
|
||||
const source = prepareHolodoriDbSource(locale, { review: true });
|
||||
const from = manifest.sources[locale].commit;
|
||||
const to = source.commit;
|
||||
// Full history is necessary: endpoint diffs alone miss changes later reverted.
|
||||
if (from !== to) {
|
||||
const shallow = git(source.repositoryPath, ['rev-parse', '--is-shallow-repository']) === 'true';
|
||||
git(source.repositoryPath, ['fetch', ...(shallow ? ['--unshallow'] : []), '--no-tags', source.repository, 'refs/heads/main']);
|
||||
try {
|
||||
git(source.repositoryPath, ['merge-base', '--is-ancestor', from, to]);
|
||||
} catch (error) {
|
||||
throw new Error(`${locale}: previous collection is not an ancestor of remote main; cannot certify a complete change report.`, { cause: error });
|
||||
}
|
||||
}
|
||||
const commits = from === to ? [] : git(source.repositoryPath, ['rev-list', '--reverse', '--topo-order', `${from}..${to}`]).split('\n').filter(Boolean);
|
||||
const entry = { from, to, repository: source.repository, commitDate: source.commitDate, commits: [], netDiff: '' };
|
||||
lines.push(`## ${locale}`, '', `기준: ${from}`, `최신: ${to}`, '');
|
||||
if (!commits.length) lines.push('변경 없음.', '');
|
||||
for (const commit of commits) {
|
||||
const title = git(source.repositoryPath, ['show', '-s', '--format=%s', commit]);
|
||||
const date = git(source.repositoryPath, ['show', '-s', '--format=%cI', commit]);
|
||||
const files = git(source.repositoryPath, ['diff-tree', '--root', '-m', '--no-commit-id', '--name-status', '-r', '--no-renames', commit]);
|
||||
const numstat = git(source.repositoryPath, ['show', '--format=', '--numstat', '--no-renames', '-m', commit]);
|
||||
const patch = git(source.repositoryPath, ['show', '--format=fuller', '--binary', '--no-ext-diff', '--no-renames', '-m', commit]);
|
||||
const parents = git(source.repositoryPath, ['show', '-s', '--format=%P', commit]).split(' ').filter(Boolean);
|
||||
const details = [];
|
||||
for (const parent of parents.length ? parents : [null]) {
|
||||
const changed = parent
|
||||
? git(source.repositoryPath, ['diff', '--name-status', '--no-renames', parent, commit])
|
||||
: files;
|
||||
for (const line of changed.split('\n').filter(Boolean)) {
|
||||
const [status, filename] = line.split('\t');
|
||||
if (!filename.endsWith('.json')) continue;
|
||||
const before = status === 'A' ? undefined : JSON.parse(git(source.repositoryPath, ['show', `${parent}:${filename}`]));
|
||||
const after = status === 'D' ? undefined : JSON.parse(git(source.repositoryPath, ['show', `${commit}:${filename}`]));
|
||||
details.push({ parent, filename, changes: jsonChanges(before, after) });
|
||||
}
|
||||
}
|
||||
entry.commits.push({ commit, title, date, files, numstat, details });
|
||||
patches.push(`Repository: ${source.repository}\n${patch}\n`);
|
||||
const fileLines = files.split('\n').filter(Boolean);
|
||||
const strings = fileLines.filter(line => /\tLang[^\t]*\.json$/.test(line));
|
||||
const masters = fileLines.filter(line => /\.json$/.test(line) && !strings.includes(line));
|
||||
const other = fileLines.filter(line => !strings.includes(line) && !masters.includes(line));
|
||||
lines.push(`### ${commit.slice(0, 8)} — ${title}`, '', `시각: ${date}`, '',
|
||||
`문자열 ${strings.length}건 · 마스터 JSON ${masters.length}건 · 기타 ${other.length}건`, '');
|
||||
for (const [label, group] of [['문자열', strings], ['마스터 JSON', masters], ['기타', other]]) {
|
||||
if (group.length) lines.push(`**${label}** (A 추가 / M 수정 / D 삭제)`, '', '```text', ...group, '```', '');
|
||||
}
|
||||
lines.push('파일별 추가·삭제 줄 수:', '', '```text', numstat, '```', '');
|
||||
for (const detail of details) {
|
||||
const counts = Object.fromEntries(['added', 'deleted', 'modified'].map(type => [type, detail.changes.filter(change => change.type === type).length]));
|
||||
lines.push(`- ${detail.filename}: 항목/필드 추가 ${counts.added}, 삭제 ${counts.deleted}, 수정 ${counts.modified}`);
|
||||
}
|
||||
lines.push('', '전체 변경 항목의 ID·필드 경로·변경 전후 값은 latest.json의 details에 기록했습니다.', '');
|
||||
}
|
||||
entry.netDiff = from === to ? '' : git(source.repositoryPath, ['diff', '--stat', from, to]);
|
||||
report.sources[locale] = entry;
|
||||
}
|
||||
lines.push('## 후속 작업', '', 'changes.patch의 전체 필드·문자열 변경을 읽고 사용자에게 의미와 영향을 요약해 먼저 제공합니다. 그 다음 계산기 데이터 확보를 진행합니다.', '',
|
||||
'계산기 반영: `npm run data:apply`. 원격 커밋이 검토 이후 바뀌면 다시 검토해야 합니다.', '');
|
||||
await mkdir(output, { recursive: true });
|
||||
await writeFile(path.join(output, 'changes.patch'), patches.join('\n'), 'utf8');
|
||||
await writeFile(path.join(output, 'latest.md'), lines.join('\n'), 'utf8');
|
||||
// Publish the machine-readable review marker only after all three locales succeed.
|
||||
await writeFile(path.join(output, 'latest.json'), `${JSON.stringify(report, null, 2)}\n`, 'utf8');
|
||||
console.log(lines.join('\n'));
|
||||
console.log(`Full changes: ${path.join(output, 'changes.patch')}`);
|
||||
@@ -1,7 +1,7 @@
|
||||
// Convert Hololive Dreams' decrypted SUS dialect to this application's compact score timeline.
|
||||
import { readFile, writeFile } from 'node:fs/promises';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { readHolodoriDbTable as readMaster } from './holodoridb-source.mjs';
|
||||
import path from 'node:path';
|
||||
import { susScoreWeight } from './sus-score-weight.mjs';
|
||||
import { formulaScoreMetadata } from './song-score-formulas.mjs';
|
||||
@@ -181,12 +181,8 @@ function parseSusTimeline(text) {
|
||||
return { songId: headers.get('MUSIC_ID'), notes, specialStartSeconds: specialStarts.sort((left, right) => left - right).map(secondsAt) };
|
||||
}
|
||||
|
||||
function readMaster(locale, filename) {
|
||||
const repository = path.resolve(`data/raw/holodoridb/${locale}`);
|
||||
return JSON.parse(execFileSync('git', ['-c', `safe.directory=${repository.replaceAll('\\', '/')}`, '-C', repository, 'show', `origin/main:${filename}`], { encoding: 'utf8', maxBuffer: 32 * 1024 * 1024 }));
|
||||
}
|
||||
const music = new Map(readMaster('kor', 'Music.json').map((row) => { const data = row.data ?? row; return [data.id, data]; }));
|
||||
const translation = new Map(readMaster('eng', 'LangMusic_Eng.json').map((row) => { const data = row.data ?? row; return [data.id, data.text]; }));
|
||||
const translation = new Map(readMaster('kor', 'LangMusic_Kor.json').map((row) => { const data = row.data ?? row; return [data.id, data.text]; }));
|
||||
const levels = new Map(readMaster('kor', 'MusicDifficulty.json').map((row) => { const data = row.data ?? row; return [`${data.musicId}.${data.difficultyType.split('_').at(-1).toLowerCase()}`, data.difficultyLevel]; }));
|
||||
const songIds = process.argv.slice(2);
|
||||
if (songIds.length === 0) throw new Error('곡 ID를 지정하세요.');
|
||||
|
||||
Reference in New Issue
Block a user