dawarich/spec/services/points/raw_data/verifier_spec.rb

167 lines
4.6 KiB
Ruby
Raw Normal View History

0.36.3 (#2013) * fix: move foreman to global gems to fix startup crash (#1971) * Update exporting code to stream points data to file in batches to red… (#1980) * Update exporting code to stream points data to file in batches to reduce memory usage * Update changelog * Update changelog * Feature/maplibre frontend (#1953) * Add a plan to use MapLibre GL JS for the frontend map rendering, replacing Leaflet * Implement phase 1 * Phases 1-3 + part of 4 * Fix e2e tests * Phase 6 * Implement fog of war * Phase 7 * Next step: fix specs, phase 7 done * Use our own map tiles * Extract v2 map logic to separate manager classes * Update settings panel on v2 map * Update v2 e2e tests structure * Reimplement location search in maps v2 * Update speed routes * Implement visits and places creation in v2 * Fix last failing test * Implement visits merging * Fix a routes e2e test and simplify the routes layer styling. * Extract js to modules from maps_v2_controller.js * Implement area creation * Fix spec problem * Fix some e2e tests * Implement live mode in v2 map * Update icons and panel * Extract some styles * Remove unused file * Start adding dark theme to popups on MapLibre maps * Make popups respect dark theme * Move v2 maps to maplibre namespace * Update v2 references to maplibre * Put place, area and visit info into side panel * Update API to use safe settings config method * Fix specs * Fix method name to config in SafeSettings and update usages accordingly * Add missing public files * Add handling for real time points * Fix remembering enabled/disabled layers of the v2 map * Fix lots of e2e tests * Add settings to select map version * Use maps/v2 as main path for MapLibre maps * Update routing * Update live mode * Update maplibre controller * Update changelog * Remove some console.log statements * Pull only necessary data for map v2 points * Feature/raw data archive (#2009) * 0.36.2 (#2007) * fix: move foreman to global gems to fix startup crash (#1971) * Update exporting code to stream points data to file in batches to red… (#1980) * Update exporting code to stream points data to file in batches to reduce memory usage * Update changelog * Update changelog * Feature/maplibre frontend (#1953) * Add a plan to use MapLibre GL JS for the frontend map rendering, replacing Leaflet * Implement phase 1 * Phases 1-3 + part of 4 * Fix e2e tests * Phase 6 * Implement fog of war * Phase 7 * Next step: fix specs, phase 7 done * Use our own map tiles * Extract v2 map logic to separate manager classes * Update settings panel on v2 map * Update v2 e2e tests structure * Reimplement location search in maps v2 * Update speed routes * Implement visits and places creation in v2 * Fix last failing test * Implement visits merging * Fix a routes e2e test and simplify the routes layer styling. * Extract js to modules from maps_v2_controller.js * Implement area creation * Fix spec problem * Fix some e2e tests * Implement live mode in v2 map * Update icons and panel * Extract some styles * Remove unused file * Start adding dark theme to popups on MapLibre maps * Make popups respect dark theme * Move v2 maps to maplibre namespace * Update v2 references to maplibre * Put place, area and visit info into side panel * Update API to use safe settings config method * Fix specs * Fix method name to config in SafeSettings and update usages accordingly * Add missing public files * Add handling for real time points * Fix remembering enabled/disabled layers of the v2 map * Fix lots of e2e tests * Add settings to select map version * Use maps/v2 as main path for MapLibre maps * Update routing * Update live mode * Update maplibre controller * Update changelog * Remove some console.log statements --------- Co-authored-by: Robin Tuszik <mail@robin.gg> * Remove esbuild scripts from package.json * Remove sideEffects field from package.json * Raw data archivation * Add tests * Fix tests * Fix tests * Update ExceptionReporter * Add schedule to run raw data archival job monthly * Change file structure for raw data archival feature * Update changelog and version for raw data archival feature --------- Co-authored-by: Robin Tuszik <mail@robin.gg> * Set raw_data to an empty hash instead of nil when archiving * Fix storage configuration and file extraction * Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation (#2018) * Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation * Remove raw data from visited cities api endpoint * Use user timezone to show dates on maps (#2020) * Fix/pre epoch time (#2019) * Use user timezone to show dates on maps * Limit timestamps to valid range to prevent database errors when users enter pre-epoch dates. * Limit timestamps to valid range to prevent database errors when users enter pre-epoch dates. * Fix tests failing due to new index on stats table * Fix failing specs * Update redis client configuration to support unix socket connection * Update changelog * Fix kml kmz import issues (#2023) * Fix kml kmz import issues * Refactor KML importer to improve readability and maintainability * Implement moving points in map v2 and fix route rendering logic to ma… (#2027) * Implement moving points in map v2 and fix route rendering logic to match map v1. * Fix route spec * fix(maplibre): update date format to ISO 8601 (#2029) * Add verification step to raw data archival process (#2028) * Add verification step to raw data archival process * Add actual verification of raw data archives after creation, and only clear raw_data for verified archives. * Fix failing specs * Eliminate zip-bomb risk * Fix potential memory leak in js * Return .keep files * Use Toast instead of alert for notifications * Add help section to navbar dropdown * Update changelog * Remove raw_data_archival_job * Ensure file is being closed properly after reading in Archivable concern --------- Co-authored-by: Robin Tuszik <mail@robin.gg>
2025-12-14 06:05:59 -05:00
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Points::RawData::Verifier do
let(:user) { create(:user) }
let(:verifier) { described_class.new }
before do
allow(PointsChannel).to receive(:broadcast_to)
end
describe '#verify_specific_archive' do
let(:test_date) { 3.months.ago.beginning_of_month.utc }
let!(:points) do
create_list(:point, 5, user: user,
timestamp: test_date.to_i,
raw_data: { lon: 13.4, lat: 52.5 })
end
let(:archive) do
# Create archive
archiver = Points::RawData::Archiver.new
archiver.archive_specific_month(user.id, test_date.year, test_date.month)
Points::RawDataArchive.last
end
it 'verifies a valid archive successfully' do
expect(archive.verified_at).to be_nil
verifier.verify_specific_archive(archive.id)
archive.reload
expect(archive.verified_at).to be_present
end
it 'detects missing file' do
archive.file.purge
archive.reload
expect do
verifier.verify_specific_archive(archive.id)
end.not_to change { archive.reload.verified_at }
end
it 'detects point count mismatch' do
# Tamper with point count
archive.update_column(:point_count, 999)
expect do
verifier.verify_specific_archive(archive.id)
end.not_to change { archive.reload.verified_at }
end
it 'detects checksum mismatch' do
# Tamper with checksum
archive.update_column(:point_ids_checksum, 'invalid')
expect do
verifier.verify_specific_archive(archive.id)
end.not_to change { archive.reload.verified_at }
end
it 'detects deleted points' do
# Force archive creation first
archive_id = archive.id
# Then delete one point from database
points.first.destroy
expect do
verifier.verify_specific_archive(archive_id)
end.not_to change { archive.reload.verified_at }
end
it 'detects raw_data mismatch between archive and database' do
# Force archive creation first
archive_id = archive.id
# Then modify raw_data in database after archiving
points.first.update_column(:raw_data, { lon: 999.0, lat: 999.0 })
expect do
verifier.verify_specific_archive(archive_id)
end.not_to change { archive.reload.verified_at }
end
it 'verifies raw_data matches between archive and database' do
# Ensure data hasn't changed
expect(points.first.raw_data).to eq({ 'lon' => 13.4, 'lat' => 52.5 })
verifier.verify_specific_archive(archive.id)
expect(archive.reload.verified_at).to be_present
end
end
describe '#verify_month' do
let(:test_date) { 3.months.ago.beginning_of_month.utc }
before do
# Create points
create_list(:point, 5, user: user,
timestamp: test_date.to_i,
raw_data: { lon: 13.4, lat: 52.5 })
# Archive them
archiver = Points::RawData::Archiver.new
archiver.archive_specific_month(user.id, test_date.year, test_date.month)
end
it 'verifies all archives for a month' do
expect(Points::RawDataArchive.where(verified_at: nil).count).to eq(1)
verifier.verify_month(user.id, test_date.year, test_date.month)
expect(Points::RawDataArchive.where(verified_at: nil).count).to eq(0)
end
end
describe '#call' do
let(:test_date) { 3.months.ago.beginning_of_month.utc }
before do
# Create points and archive
create_list(:point, 5, user: user,
timestamp: test_date.to_i,
raw_data: { lon: 13.4, lat: 52.5 })
archiver = Points::RawData::Archiver.new
archiver.archive_specific_month(user.id, test_date.year, test_date.month)
end
it 'verifies all unverified archives' do
expect(Points::RawDataArchive.where(verified_at: nil).count).to eq(1)
result = verifier.call
expect(result[:verified]).to eq(1)
expect(result[:failed]).to eq(0)
expect(Points::RawDataArchive.where(verified_at: nil).count).to eq(0)
end
it 'reports failures' do
# Tamper with archive
Points::RawDataArchive.last.update_column(:point_count, 999)
result = verifier.call
expect(result[:verified]).to eq(0)
expect(result[:failed]).to eq(1)
end
it 'skips already verified archives' do
# Verify once
verifier.call
# Try to verify again with a new verifier instance
new_verifier = Points::RawData::Verifier.new
result = new_verifier.call
expect(result[:verified]).to eq(0)
expect(result[:failed]).to eq(0)
end
end
end