mirror of
https://github.com/Freika/dawarich.git
synced 2026-01-09 16:57:12 -05:00
* fix: move foreman to global gems to fix startup crash (#1971) * Update exporting code to stream points data to file in batches to red… (#1980) * Update exporting code to stream points data to file in batches to reduce memory usage * Update changelog * Update changelog * Feature/maplibre frontend (#1953) * Add a plan to use MapLibre GL JS for the frontend map rendering, replacing Leaflet * Implement phase 1 * Phases 1-3 + part of 4 * Fix e2e tests * Phase 6 * Implement fog of war * Phase 7 * Next step: fix specs, phase 7 done * Use our own map tiles * Extract v2 map logic to separate manager classes * Update settings panel on v2 map * Update v2 e2e tests structure * Reimplement location search in maps v2 * Update speed routes * Implement visits and places creation in v2 * Fix last failing test * Implement visits merging * Fix a routes e2e test and simplify the routes layer styling. * Extract js to modules from maps_v2_controller.js * Implement area creation * Fix spec problem * Fix some e2e tests * Implement live mode in v2 map * Update icons and panel * Extract some styles * Remove unused file * Start adding dark theme to popups on MapLibre maps * Make popups respect dark theme * Move v2 maps to maplibre namespace * Update v2 references to maplibre * Put place, area and visit info into side panel * Update API to use safe settings config method * Fix specs * Fix method name to config in SafeSettings and update usages accordingly * Add missing public files * Add handling for real time points * Fix remembering enabled/disabled layers of the v2 map * Fix lots of e2e tests * Add settings to select map version * Use maps/v2 as main path for MapLibre maps * Update routing * Update live mode * Update maplibre controller * Update changelog * Remove some console.log statements * Pull only necessary data for map v2 points * Feature/raw data archive (#2009) * 0.36.2 (#2007) * fix: move foreman to global gems to fix startup crash (#1971) * Update exporting code to stream points data to file in batches to red… (#1980) * Update exporting code to stream points data to file in batches to reduce memory usage * Update changelog * Update changelog * Feature/maplibre frontend (#1953) * Add a plan to use MapLibre GL JS for the frontend map rendering, replacing Leaflet * Implement phase 1 * Phases 1-3 + part of 4 * Fix e2e tests * Phase 6 * Implement fog of war * Phase 7 * Next step: fix specs, phase 7 done * Use our own map tiles * Extract v2 map logic to separate manager classes * Update settings panel on v2 map * Update v2 e2e tests structure * Reimplement location search in maps v2 * Update speed routes * Implement visits and places creation in v2 * Fix last failing test * Implement visits merging * Fix a routes e2e test and simplify the routes layer styling. * Extract js to modules from maps_v2_controller.js * Implement area creation * Fix spec problem * Fix some e2e tests * Implement live mode in v2 map * Update icons and panel * Extract some styles * Remove unused file * Start adding dark theme to popups on MapLibre maps * Make popups respect dark theme * Move v2 maps to maplibre namespace * Update v2 references to maplibre * Put place, area and visit info into side panel * Update API to use safe settings config method * Fix specs * Fix method name to config in SafeSettings and update usages accordingly * Add missing public files * Add handling for real time points * Fix remembering enabled/disabled layers of the v2 map * Fix lots of e2e tests * Add settings to select map version * Use maps/v2 as main path for MapLibre maps * Update routing * Update live mode * Update maplibre controller * Update changelog * Remove some console.log statements --------- Co-authored-by: Robin Tuszik <mail@robin.gg> * Remove esbuild scripts from package.json * Remove sideEffects field from package.json * Raw data archivation * Add tests * Fix tests * Fix tests * Update ExceptionReporter * Add schedule to run raw data archival job monthly * Change file structure for raw data archival feature * Update changelog and version for raw data archival feature --------- Co-authored-by: Robin Tuszik <mail@robin.gg> * Set raw_data to an empty hash instead of nil when archiving * Fix storage configuration and file extraction * Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation (#2018) * Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation * Remove raw data from visited cities api endpoint * Use user timezone to show dates on maps (#2020) * Fix/pre epoch time (#2019) * Use user timezone to show dates on maps * Limit timestamps to valid range to prevent database errors when users enter pre-epoch dates. * Limit timestamps to valid range to prevent database errors when users enter pre-epoch dates. * Fix tests failing due to new index on stats table * Fix failing specs * Update redis client configuration to support unix socket connection * Update changelog * Fix kml kmz import issues (#2023) * Fix kml kmz import issues * Refactor KML importer to improve readability and maintainability * Implement moving points in map v2 and fix route rendering logic to ma… (#2027) * Implement moving points in map v2 and fix route rendering logic to match map v1. * Fix route spec * fix(maplibre): update date format to ISO 8601 (#2029) * Add verification step to raw data archival process (#2028) * Add verification step to raw data archival process * Add actual verification of raw data archives after creation, and only clear raw_data for verified archives. * Fix failing specs * Eliminate zip-bomb risk * Fix potential memory leak in js * Return .keep files * Use Toast instead of alert for notifications * Add help section to navbar dropdown * Update changelog * Remove raw_data_archival_job * Ensure file is being closed properly after reading in Archivable concern * Add composite index to stats table if not exists * Update changelog * Update entrypoint to always sync static assets (not only new ones) * Add family layer to MapLibre maps (#2055) * Add family layer to MapLibre maps * Update migration * Don't show family toggle if feature is disabled * Update changelog * Return changelog * Update changelog * Update tailwind file * Bump sentry-rails from 6.0.0 to 6.1.0 (#1945) Bumps [sentry-rails](https://github.com/getsentry/sentry-ruby) from 6.0.0 to 6.1.0. - [Release notes](https://github.com/getsentry/sentry-ruby/releases) - [Changelog](https://github.com/getsentry/sentry-ruby/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-ruby/compare/6.0.0...6.1.0) --- updated-dependencies: - dependency-name: sentry-rails dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump turbo-rails from 2.0.17 to 2.0.20 (#1944) Bumps [turbo-rails](https://github.com/hotwired/turbo-rails) from 2.0.17 to 2.0.20. - [Release notes](https://github.com/hotwired/turbo-rails/releases) - [Commits](https://github.com/hotwired/turbo-rails/compare/v2.0.17...v2.0.20) --- updated-dependencies: - dependency-name: turbo-rails dependency-version: 2.0.20 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Evgenii Burmakin <Freika@users.noreply.github.com> * Bump webmock from 3.25.1 to 3.26.1 (#1943) Bumps [webmock](https://github.com/bblimke/webmock) from 3.25.1 to 3.26.1. - [Release notes](https://github.com/bblimke/webmock/releases) - [Changelog](https://github.com/bblimke/webmock/blob/master/CHANGELOG.md) - [Commits](https://github.com/bblimke/webmock/compare/v3.25.1...v3.26.1) --- updated-dependencies: - dependency-name: webmock dependency-version: 3.26.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Evgenii Burmakin <Freika@users.noreply.github.com> * Bump brakeman from 7.1.0 to 7.1.1 (#1942) Bumps [brakeman](https://github.com/presidentbeef/brakeman) from 7.1.0 to 7.1.1. - [Release notes](https://github.com/presidentbeef/brakeman/releases) - [Changelog](https://github.com/presidentbeef/brakeman/blob/main/CHANGES.md) - [Commits](https://github.com/presidentbeef/brakeman/compare/v7.1.0...v7.1.1) --- updated-dependencies: - dependency-name: brakeman dependency-version: 7.1.1 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump redis from 5.4.0 to 5.4.1 (#1941) Bumps [redis](https://github.com/redis/redis-rb) from 5.4.0 to 5.4.1. - [Changelog](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md) - [Commits](https://github.com/redis/redis-rb/compare/v5.4.0...v5.4.1) --- updated-dependencies: - dependency-name: redis dependency-version: 5.4.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Put import deletion into background job (#2045) * Put import deletion into background job * Update changelog * fix null type error and update heatmap styling (#2037) * fix: use constant weight for maplibre heatmap layer * fix null type, update heatmap styling * improve heatmap styling * fix typo * Fix stats calculation to recursively reduce H3 resolution when too ma… (#2065) * Fix stats calculation to recursively reduce H3 resolution when too many hexagons are generated * Update CHANGELOG.md * Validate trip start and end dates (#2066) * Validate trip start and end dates * Update changelog * Update migration to clean up duplicate stats before adding unique index * Fix fog of war radius setting being ignored and applying settings causing errors (#2068) * Update changelog * Add Rack::Deflater middleware to config/application.rb to enable gzip compression for responses. * Add composite index to points on user_id and timestamp * Deduplicte points based on timestamp brought to unix time * Fix/stats cache invalidation (#2072) * Fix family layer toggle in Map v2 settings for non-selfhosted env * Invalidate cache * Remove comments * Remove comment * Add new indicies to improve performance and remove unused ones to opt… (#2078) * Add new indicies to improve performance and remove unused ones to optimize database. * Remove comments * Update map search suggestions panel styling * Add yearly digest (#2073) * Add yearly digest * Rename YearlyDigests to Users::Digests * Minor changes * Update yearly digest layout and styles * Add flags and chart to email * Update colors * Fix layout of stats in yearly digest view * Remove cron job for yearly digest scheduling * Update CHANGELOG.md * Update digest email setting handling * Allow sharing digest for 1 week or 1 month * Change Digests Distance to Bigint * Fix settings page * Update changelog * Add RailsPulse (#2079) * Add RailsPulse * Add RailsPulse monitoring tool with basic HTTP authentication * Bring points_count to integer * Update migration and version * Update rubocop issues --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Robin Tuszik <mail@robin.gg> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
133 lines
8.2 KiB
Ruby
133 lines
8.2 KiB
Ruby
# Rails Pulse Database Schema
|
|
# This file contains the complete schema for Rails Pulse tables
|
|
# Load with: rails db:schema:load:rails_pulse or db:prepare
|
|
|
|
RailsPulse::Schema = lambda do |connection|
|
|
# Skip if all tables already exist to prevent conflicts
|
|
required_tables = [ :rails_pulse_routes, :rails_pulse_queries, :rails_pulse_requests, :rails_pulse_operations, :rails_pulse_summaries ]
|
|
|
|
if ENV["CI"] == "true"
|
|
existing_tables = required_tables.select { |table| connection.table_exists?(table) }
|
|
missing_tables = required_tables - existing_tables
|
|
puts "[RailsPulse::Schema] Existing tables: #{existing_tables.join(', ')}" if existing_tables.any?
|
|
puts "[RailsPulse::Schema] Missing tables: #{missing_tables.join(', ')}" if missing_tables.any?
|
|
end
|
|
|
|
return if required_tables.all? { |table| connection.table_exists?(table) }
|
|
|
|
connection.create_table :rails_pulse_routes do |t|
|
|
t.string :method, null: false, comment: "HTTP method (e.g., GET, POST)"
|
|
t.string :path, null: false, comment: "Request path (e.g., /posts/index)"
|
|
t.text :tags, comment: "JSON array of tags for filtering and categorization"
|
|
t.timestamps
|
|
end
|
|
|
|
connection.add_index :rails_pulse_routes, [ :method, :path ], unique: true, name: "index_rails_pulse_routes_on_method_and_path"
|
|
|
|
connection.create_table :rails_pulse_queries do |t|
|
|
t.string :normalized_sql, limit: 1000, null: false, comment: "Normalized SQL query string (e.g., SELECT * FROM users WHERE id = ?)"
|
|
t.datetime :analyzed_at, comment: "When query analysis was last performed"
|
|
t.text :explain_plan, comment: "EXPLAIN output from actual SQL execution"
|
|
t.text :issues, comment: "JSON array of detected performance issues"
|
|
t.text :metadata, comment: "JSON object containing query complexity metrics"
|
|
t.text :query_stats, comment: "JSON object with query characteristics analysis"
|
|
t.text :backtrace_analysis, comment: "JSON object with call chain and N+1 detection"
|
|
t.text :index_recommendations, comment: "JSON array of database index recommendations"
|
|
t.text :n_plus_one_analysis, comment: "JSON object with enhanced N+1 query detection results"
|
|
t.text :suggestions, comment: "JSON array of optimization recommendations"
|
|
t.text :tags, comment: "JSON array of tags for filtering and categorization"
|
|
t.timestamps
|
|
end
|
|
|
|
connection.add_index :rails_pulse_queries, :normalized_sql, unique: true, name: "index_rails_pulse_queries_on_normalized_sql", length: 191
|
|
|
|
connection.create_table :rails_pulse_requests do |t|
|
|
t.references :route, null: false, foreign_key: { to_table: :rails_pulse_routes }, comment: "Link to the route"
|
|
t.decimal :duration, precision: 15, scale: 6, null: false, comment: "Total request duration in milliseconds"
|
|
t.integer :status, null: false, comment: "HTTP status code (e.g., 200, 500)"
|
|
t.boolean :is_error, null: false, default: false, comment: "True if status >= 500"
|
|
t.string :request_uuid, null: false, comment: "Unique identifier for the request (e.g., UUID)"
|
|
t.string :controller_action, comment: "Controller and action handling the request (e.g., PostsController#show)"
|
|
t.timestamp :occurred_at, null: false, comment: "When the request started"
|
|
t.text :tags, comment: "JSON array of tags for filtering and categorization"
|
|
t.timestamps
|
|
end
|
|
|
|
connection.add_index :rails_pulse_requests, :occurred_at, name: "index_rails_pulse_requests_on_occurred_at"
|
|
connection.add_index :rails_pulse_requests, :request_uuid, unique: true, name: "index_rails_pulse_requests_on_request_uuid"
|
|
connection.add_index :rails_pulse_requests, [ :route_id, :occurred_at ], name: "index_rails_pulse_requests_on_route_id_and_occurred_at"
|
|
|
|
connection.create_table :rails_pulse_operations do |t|
|
|
t.references :request, null: false, foreign_key: { to_table: :rails_pulse_requests }, comment: "Link to the request"
|
|
t.references :query, foreign_key: { to_table: :rails_pulse_queries }, index: true, comment: "Link to the normalized SQL query"
|
|
t.string :operation_type, null: false, comment: "Type of operation (e.g., database, view, gem_call)"
|
|
t.string :label, null: false, comment: "Descriptive name (e.g., SELECT FROM users WHERE id = 1, render layout)"
|
|
t.decimal :duration, precision: 15, scale: 6, null: false, comment: "Operation duration in milliseconds"
|
|
t.string :codebase_location, comment: "File and line number (e.g., app/models/user.rb:25)"
|
|
t.float :start_time, null: false, default: 0.0, comment: "Operation start time in milliseconds"
|
|
t.timestamp :occurred_at, null: false, comment: "When the request started"
|
|
t.timestamps
|
|
end
|
|
|
|
connection.add_index :rails_pulse_operations, :operation_type, name: "index_rails_pulse_operations_on_operation_type"
|
|
connection.add_index :rails_pulse_operations, :occurred_at, name: "index_rails_pulse_operations_on_occurred_at"
|
|
connection.add_index :rails_pulse_operations, [ :query_id, :occurred_at ], name: "index_rails_pulse_operations_on_query_and_time"
|
|
connection.add_index :rails_pulse_operations, [ :query_id, :duration, :occurred_at ], name: "index_rails_pulse_operations_query_performance"
|
|
connection.add_index :rails_pulse_operations, [ :occurred_at, :duration, :operation_type ], name: "index_rails_pulse_operations_on_time_duration_type"
|
|
|
|
connection.create_table :rails_pulse_summaries do |t|
|
|
# Time fields
|
|
t.datetime :period_start, null: false, comment: "Start of the aggregation period"
|
|
t.datetime :period_end, null: false, comment: "End of the aggregation period"
|
|
t.string :period_type, null: false, comment: "Aggregation period type: hour, day, week, month"
|
|
|
|
# Polymorphic association to handle both routes and queries
|
|
t.references :summarizable, polymorphic: true, null: false, index: true, comment: "Link to Route or Query"
|
|
# This creates summarizable_type (e.g., 'RailsPulse::Route', 'RailsPulse::Query')
|
|
# and summarizable_id (route_id or query_id)
|
|
|
|
# Universal metrics
|
|
t.integer :count, default: 0, null: false, comment: "Total number of requests/operations"
|
|
t.float :avg_duration, comment: "Average duration in milliseconds"
|
|
t.float :min_duration, comment: "Minimum duration in milliseconds"
|
|
t.float :max_duration, comment: "Maximum duration in milliseconds"
|
|
t.float :p50_duration, comment: "50th percentile duration"
|
|
t.float :p95_duration, comment: "95th percentile duration"
|
|
t.float :p99_duration, comment: "99th percentile duration"
|
|
t.float :total_duration, comment: "Total duration in milliseconds"
|
|
t.float :stddev_duration, comment: "Standard deviation of duration"
|
|
|
|
# Request/Route specific metrics
|
|
t.integer :error_count, default: 0, comment: "Number of error responses (5xx)"
|
|
t.integer :success_count, default: 0, comment: "Number of successful responses"
|
|
t.integer :status_2xx, default: 0, comment: "Number of 2xx responses"
|
|
t.integer :status_3xx, default: 0, comment: "Number of 3xx responses"
|
|
t.integer :status_4xx, default: 0, comment: "Number of 4xx responses"
|
|
t.integer :status_5xx, default: 0, comment: "Number of 5xx responses"
|
|
|
|
t.timestamps
|
|
end
|
|
|
|
# Unique constraint and indexes for summaries
|
|
connection.add_index :rails_pulse_summaries, [ :summarizable_type, :summarizable_id, :period_type, :period_start ],
|
|
unique: true,
|
|
name: "idx_pulse_summaries_unique"
|
|
connection.add_index :rails_pulse_summaries, [ :period_type, :period_start ], name: "index_rails_pulse_summaries_on_period"
|
|
connection.add_index :rails_pulse_summaries, :created_at, name: "index_rails_pulse_summaries_on_created_at"
|
|
|
|
# Add indexes to existing tables for efficient aggregation
|
|
connection.add_index :rails_pulse_requests, [ :created_at, :route_id ], name: "idx_requests_for_aggregation"
|
|
connection.add_index :rails_pulse_requests, :created_at, name: "idx_requests_created_at"
|
|
|
|
connection.add_index :rails_pulse_operations, [ :created_at, :query_id ], name: "idx_operations_for_aggregation"
|
|
connection.add_index :rails_pulse_operations, :created_at, name: "idx_operations_created_at"
|
|
|
|
if ENV["CI"] == "true"
|
|
created_tables = required_tables.select { |table| connection.table_exists?(table) }
|
|
puts "[RailsPulse::Schema] Successfully created tables: #{created_tables.join(', ')}"
|
|
end
|
|
end
|
|
|
|
if defined?(RailsPulse::ApplicationRecord)
|
|
RailsPulse::Schema.call(RailsPulse::ApplicationRecord.connection)
|
|
end
|