Merge remote-tracking branch 'origin/master' into fix/reverse-geocoded-points

This commit is contained in:
Eugene Burmakin 2024-12-10 20:15:42 +01:00
commit 9bca8cfeaa
93 changed files with 2554 additions and 658 deletions

View file

@ -1 +1 @@
0.18.3
0.19.4

View file

@ -5,16 +5,124 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
# 0.18.3 - 2024-12-02
# 0.19.4 - 2024-12-10
⚠️ This release introduces a breaking change. ⚠️
The `GET /api/v1/trips/:id/photos` endpoint now returns a different structure of the response:
```diff
{
id: 1,
latitude: 10,
longitude: 10,
localDateTime: "2024-01-01T00:00:00Z",
originalFileName: "photo.jpg",
city: "Berlin",
state: "Berlin",
country: "Germany",
type: "image",
+ orientation: "portrait",
source: "photoprism"
}
```
### Fixed
- Fixed a bug where the Photoprism photos were not being shown on the trip page.
- Fixed a bug where the Immich photos were not being shown on the trip page.
- Fixed a bug where the route popup was showing distance in kilometers instead of miles. #490
### Added
- A link to the Photoprism photos on the trip page if there are any.
- A `orientation` field in the Api::PhotoSerializer, hence the `GET /api/v1/photos` endpoint now includes the orientation of the photo. Valid values are `portrait` and `landscape`.
- Examples for the `type`, `orientation` and `source` fields in the `GET /api/v1/photos` endpoint in the Swagger UI.
- `DISABLE_TELEMETRY` env var to disable telemetry. More on telemetry: https://dawarich.app/docs/tutorials/telemetry
- `reverse_geocoded_at` column added to the `points` table.
### Changed
- On the Stats page, the "Reverse geocoding" section is now showing the number of points that were reverse geocoded based on `reverse_geocoded_at` column, value of which is based on the time when the point was reverse geocoded. If no geodata for the point is available, `reverse_geocoded_at` will be set anyway. Number of points that were reverse geocoded but no geodata is available for them is shown below the "Reverse geocoded" number.
# 0.19.3 - 2024-12-06
### Changed
- Refactored stats calculation to calculate only necessary stats, instead of calculating all stats
- Stats are now being calculated every 1 hour instead of 6 hours
- List of years on the Map page is now being calculated based on user's points instead of stats. It's also being cached for 1 day due to the fact that it's usually a heavy operation based on the number of points.
- Reverse-geocoding points is now being performed in batches of 1,000 points to prevent memory exhaustion.
### Added
- In-app notification about telemetry being enabled.
# 0.19.2 - 2024-12-04
## The Telemetry release
Dawarich now can collect usage metrics and send them to InfluxDB. Before this release, the only metrics that could be somehow tracked by developers (only @Freika, as of now) were the number of stars on GitHub and the overall number of docker images being pulled, across all versions of Dawarich, non-splittable by version. New in-app telemetry will allow us to track more granular metrics, allowing me to make decisions based on facts, not just guesses.
I'm aware about the privacy concerns, so I want to be very transparent about what data is being sent and how it's used.
Data being sent:
- Number of DAU (Daily Active Users)
- App version
- Instance ID (unique identifier of the Dawarich instance built by hashing the api key of the first user in the database)
The data is being sent to a InfluxDB instance hosted by me and won't be shared with anyone.
Basically this set of metrics allows me to see how many people are using Dawarich and what versions they are using. No other data is being sent, nor it gives me any knowledge about individual users or their data or activity.
The telemetry is enabled by default, but it **can be disabled** by setting `DISABLE_TELEMETRY` env var to `true`. The dataset might change in the future, but any changes will be documented here in the changelog and in every release as well as on the [telemetry page](https://dawarich.app/docs/tutorials/telemetry) of the website docs.
### Added
- Telemetry feature. It's now collecting usage metrics and sending them to InfluxDB.
# 0.19.1 - 2024-12-04
### Fixed
- Sidekiq is now being correctly exported to Prometheus with `PROMETHEUS_EXPORTER_ENABLED=true` env var in `dawarich_sidekiq` service.
# 0.19.0 - 2024-12-04
## The Photoprism integration release
⚠️ This release introduces a breaking change. ⚠️
The `GET /api/v1/photos` endpoint now returns following structure of the response:
```json
[
{
"id": "1",
"latitude": 11.22,
"longitude": 12.33,
"localDateTime": "2024-01-01T00:00:00Z",
"originalFileName": "photo.jpg",
"city": "Berlin",
"state": "Berlin",
"country": "Germany",
"type": "image", // "image" or "video"
"source": "photoprism" // "photoprism" or "immich"
}
]
```
### Added
- Photos from Photoprism are now can be shown on the map. To enable this feature, you need to provide your Photoprism instance URL and API key in the Settings page. Then you need to enable "Photos" layer on the map (top right corner).
- Geodata is now can be imported from Photoprism to Dawarich. The "Import Photoprism data" button on the Imports page will start the import process.
### Fixed
- z-index on maps so they won't overlay notifications dropdown
- Redis connectivity where it's not required
# 0.18.2 - 2024-11-29
### Added

View file

@ -236,7 +236,7 @@ GEM
patience_diff (1.2.0)
optimist (~> 3.0)
pg (1.5.9)
prometheus_exporter (2.1.1)
prometheus_exporter (2.2.0)
webrick
pry (0.14.2)
coderay (~> 1.1)
@ -417,7 +417,7 @@ GEM
addressable (>= 2.8.0)
crack (>= 0.3.2)
hashdiff (>= 0.4.0, < 2.0.0)
webrick (1.9.0)
webrick (1.9.1)
websocket-driver (0.7.6)
websocket-extensions (>= 0.1.0)
websocket-extensions (0.1.5)

View file

@ -99,6 +99,10 @@ Simply install one of the supported apps on your device and configure it to send
### 📊 Statistics
- Analyze your travel history: number of countries/cities visited, distance traveled, and time spent, broken down by year and month.
### 📸 Integrations
- Provide credentials for Immich or Photoprism (or both!) and Dawarich will automatically import geodata from your photos.
- You'll also be able to visualize your photos on the map!
### 📥 Import Your Data
- Import from various sources:
- Google Maps Timeline

File diff suppressed because one or more lines are too long

View file

@ -1,38 +1,52 @@
# frozen_string_literal: true
class Api::V1::PhotosController < ApiController
before_action :check_integration_configured, only: %i[index thumbnail]
before_action :check_source, only: %i[thumbnail]
def index
@photos = Rails.cache.fetch("photos_#{params[:start_date]}_#{params[:end_date]}", expires_in: 1.day) do
Immich::RequestPhotos.new(
current_api_user,
start_date: params[:start_date],
end_date: params[:end_date]
).call.reject { |asset| asset['type'].downcase == 'video' }
Photos::Search.new(current_api_user, start_date: params[:start_date], end_date: params[:end_date]).call
end
render json: @photos, status: :ok
end
def thumbnail
response = Rails.cache.fetch("photo_thumbnail_#{params[:id]}", expires_in: 1.day) do
HTTParty.get(
"#{current_api_user.settings['immich_url']}/api/assets/#{params[:id]}/thumbnail?size=preview",
headers: {
'x-api-key' => current_api_user.settings['immich_api_key'],
'accept' => 'application/octet-stream'
}
)
end
response = fetch_cached_thumbnail(params[:source])
handle_thumbnail_response(response)
end
private
def fetch_cached_thumbnail(source)
Rails.cache.fetch("photo_thumbnail_#{params[:id]}", expires_in: 1.day) do
Photos::Thumbnail.new(current_api_user, source, params[:id]).call
end
end
def handle_thumbnail_response(response)
if response.success?
send_data(
response.body,
type: 'image/jpeg',
disposition: 'inline',
status: :ok
)
send_data(response.body, type: 'image/jpeg', disposition: 'inline', status: :ok)
else
render json: { error: 'Failed to fetch thumbnail' }, status: response.code
end
end
def integration_configured?
current_api_user.immich_integration_configured? || current_api_user.photoprism_integration_configured?
end
def check_integration_configured
unauthorized_integration unless integration_configured?
end
def check_source
unauthorized_integration unless params[:source] == 'immich' || params[:source] == 'photoprism'
end
def unauthorized_integration
render json: { error: "#{params[:source]&.capitalize} integration not configured" },
status: :unauthorized
end
end

View file

@ -12,16 +12,11 @@ class Api::V1::SettingsController < ApiController
settings_params.each { |key, value| current_api_user.settings[key] = value }
if current_api_user.save
render json: {
message: 'Settings updated',
settings: current_api_user.settings,
status: 'success'
}, status: :ok
render json: { message: 'Settings updated', settings: current_api_user.settings, status: 'success' },
status: :ok
else
render json: {
message: 'Something went wrong',
errors: current_api_user.errors.full_messages
}, status: :unprocessable_entity
render json: { message: 'Something went wrong', errors: current_api_user.errors.full_messages },
status: :unprocessable_entity
end
end
@ -31,7 +26,8 @@ class Api::V1::SettingsController < ApiController
params.require(:settings).permit(
:meters_between_routes, :minutes_between_routes, :fog_of_war_meters,
:time_threshold_minutes, :merge_threshold_minutes, :route_opacity,
:preferred_map_layer, :points_rendering_mode, :live_map_enabled
:preferred_map_layer, :points_rendering_mode, :live_map_enabled,
:immich_url, :immich_api_key, :photoprism_url, :photoprism_api_key
)
end
end

View file

@ -36,9 +36,7 @@ class MapController < ApplicationController
@distance ||= 0
@coordinates.each_cons(2) do
@distance += Geocoder::Calculations.distance_between(
[_1[0], _1[1]], [_2[0], _2[1]], units: DISTANCE_UNIT
)
@distance += DistanceCalculator.new([_1[0], _1[1]], [_2[0], _2[1]]).call
end
@distance.round(1)

View file

@ -31,7 +31,7 @@ class SettingsController < ApplicationController
params.require(:settings).permit(
:meters_between_routes, :minutes_between_routes, :fog_of_war_meters,
:time_threshold_minutes, :merge_threshold_minutes, :route_opacity,
:immich_url, :immich_api_key
:immich_url, :immich_api_key, :photoprism_url, :photoprism_api_key
)
end
end

View file

@ -16,7 +16,11 @@ class StatsController < ApplicationController
end
def update
Stats::CalculatingJob.perform_later(current_user.id)
current_user.years_tracked.each do |year|
(1..12).each do |month|
Stats::CalculatingJob.perform_later(current_user.id, year, month)
end
end
redirect_to stats_path, notice: 'Stats are being updated', status: :see_other
end

View file

@ -15,9 +15,10 @@ class TripsController < ApplicationController
:country
).map { [_1.to_f, _2.to_f, _3.to_s, _4.to_s, _5.to_s, _6.to_s, _7.to_s, _8.to_s] }
@photos = Rails.cache.fetch("trip_photos_#{@trip.id}", expires_in: 1.day) do
@trip.photos
@photo_previews = Rails.cache.fetch("trip_photos_#{@trip.id}", expires_in: 1.day) do
@trip.photo_previews
end
@photo_sources = @trip.photo_sources
end
def new

View file

@ -110,14 +110,4 @@ module ApplicationHelper
def human_date(date)
date.strftime('%e %B %Y')
end
def immich_search_url(base_url, start_date, end_date)
query = {
takenAfter: "#{start_date.to_date}T00:00:00.000Z",
takenBefore: "#{end_date.to_date}T23:59:59.999Z"
}
encoded_query = URI.encode_www_form_component(query.to_json)
"#{base_url}/search?query=#{encoded_query}"
end
end

View file

@ -0,0 +1,26 @@
# frozen_string_literal: true
module TripsHelper
def immich_search_url(base_url, start_date, end_date)
query = {
takenAfter: "#{start_date.to_date}T00:00:00.000Z",
takenBefore: "#{end_date.to_date}T23:59:59.999Z"
}
encoded_query = URI.encode_www_form_component(query.to_json)
"#{base_url}/search?query=#{encoded_query}"
end
def photoprism_search_url(base_url, start_date, _end_date)
"#{base_url}/library/browse?view=cards&year=#{start_date.year}&month=#{start_date.month}&order=newest&public=true&quality=3"
end
def photo_search_url(source, settings, start_date, end_date)
case source
when 'immich'
immich_search_url(settings['immich_url'], start_date, end_date)
when 'photoprism'
photoprism_search_url(settings['photoprism_url'], start_date, end_date)
end
end
end

View file

@ -62,7 +62,7 @@ export default class extends Controller {
this.markersLayer = L.layerGroup(this.markersArray);
this.heatmapMarkers = this.markersArray.map((element) => [element._latlng.lat, element._latlng.lng, 0.2]);
this.polylinesLayer = createPolylinesLayer(this.markers, this.map, this.timezone, this.routeOpacity, this.userSettings);
this.polylinesLayer = createPolylinesLayer(this.markers, this.map, this.timezone, this.routeOpacity, this.userSettings, this.distanceUnit);
this.heatmapLayer = L.heatLayer(this.heatmapMarkers, { radius: 20 }).addTo(this.map);
this.fogOverlay = L.layerGroup(); // Initialize fog layer
this.areasLayer = L.layerGroup(); // Initialize areas layer
@ -137,10 +137,13 @@ export default class extends Controller {
this.map.addControl(this.drawControl);
}
if (e.name === 'Photos') {
if (!this.userSettings.immich_url || !this.userSettings.immich_api_key) {
if (
(!this.userSettings.immich_url || !this.userSettings.immich_api_key) &&
(!this.userSettings.photoprism_url || !this.userSettings.photoprism_api_key)
) {
showFlashMessage(
'error',
'Immich integration is not configured. Please check your settings.'
'Photos integration is not configured. Please check your integrations settings.'
);
return;
}
@ -212,7 +215,8 @@ export default class extends Controller {
this.map,
this.timezone,
this.routeOpacity,
this.userSettings
this.userSettings,
this.distanceUnit
);
// Pan map to new location
@ -684,7 +688,7 @@ export default class extends Controller {
// Recreate layers only if they don't exist
this.markersLayer = preserveLayers.Points || L.layerGroup(createMarkersArray(this.markers, newSettings));
this.polylinesLayer = preserveLayers.Polylines || createPolylinesLayer(this.markers, this.map, this.timezone, this.routeOpacity, this.userSettings);
this.polylinesLayer = preserveLayers.Polylines || createPolylinesLayer(this.markers, this.map, this.timezone, this.routeOpacity, this.userSettings, this.distanceUnit);
this.heatmapLayer = preserveLayers.Heatmap || L.heatLayer(this.markers.map((element) => [element[0], element[1], 0.2]), { radius: 20 });
this.fogOverlay = preserveLayers["Fog of War"] || L.layerGroup();
this.areasLayer = preserveLayers.Areas || L.layerGroup();
@ -799,7 +803,7 @@ export default class extends Controller {
createPhotoMarker(photo) {
if (!photo.exifInfo?.latitude || !photo.exifInfo?.longitude) return;
const thumbnailUrl = `/api/v1/photos/${photo.id}/thumbnail.jpg?api_key=${this.apiKey}`;
const thumbnailUrl = `/api/v1/photos/${photo.id}/thumbnail.jpg?api_key=${this.apiKey}&source=${photo.source}`;
const icon = L.divIcon({
className: 'photo-marker',
@ -836,7 +840,7 @@ export default class extends Controller {
<h3 class="font-bold">${photo.originalFileName}</h3>
<p>Taken: ${new Date(photo.localDateTime).toLocaleString()}</p>
<p>Location: ${photo.exifInfo.city}, ${photo.exifInfo.state}, ${photo.exifInfo.country}</p>
${photo.type === 'VIDEO' ? '🎥 Video' : '📷 Photo'}
${photo.type === 'video' ? '🎥 Video' : '📷 Photo'}
</div>
`;
marker.bindPopup(popupContent);

View file

@ -80,10 +80,10 @@ export default class extends Controller {
this.map.on('overlayadd', (e) => {
if (e.name !== 'Photos') return;
if (!this.userSettings.immich_url || !this.userSettings.immich_api_key) {
if ((!this.userSettings.immich_url || !this.userSettings.immich_api_key) && (!this.userSettings.photoprism_url || !this.userSettings.photoprism_api_key)) {
showFlashMessage(
'error',
'Immich integration is not configured. Please check your settings.'
'Photos integration is not configured. Please check your integrations settings.'
);
return;
}

View file

@ -162,7 +162,7 @@ export async function fetchAndDisplayPhotos({ map, photoMarkers, apiKey, startDa
const response = await fetch(`/api/v1/photos?${params}`);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
throw new Error(`HTTP error! status: ${response.status}, response: ${response.body}`);
}
const photos = await response.json();
@ -171,10 +171,10 @@ export async function fetchAndDisplayPhotos({ map, photoMarkers, apiKey, startDa
const photoLoadPromises = photos.map(photo => {
return new Promise((resolve) => {
const img = new Image();
const thumbnailUrl = `/api/v1/photos/${photo.id}/thumbnail.jpg?api_key=${apiKey}`;
const thumbnailUrl = `/api/v1/photos/${photo.id}/thumbnail.jpg?api_key=${apiKey}&source=${photo.source}`;
img.onload = () => {
createPhotoMarker(photo, userSettings.immich_url, photoMarkers, apiKey);
createPhotoMarker(photo, userSettings, photoMarkers, apiKey);
resolve();
};
@ -216,11 +216,44 @@ export async function fetchAndDisplayPhotos({ map, photoMarkers, apiKey, startDa
}
}
function getPhotoLink(photo, userSettings) {
switch (photo.source) {
case 'immich':
const startOfDay = new Date(photo.localDateTime);
startOfDay.setHours(0, 0, 0, 0);
export function createPhotoMarker(photo, immichUrl, photoMarkers,apiKey) {
if (!photo.exifInfo?.latitude || !photo.exifInfo?.longitude) return;
const endOfDay = new Date(photo.localDateTime);
endOfDay.setHours(23, 59, 59, 999);
const thumbnailUrl = `/api/v1/photos/${photo.id}/thumbnail.jpg?api_key=${apiKey}`;
const queryParams = {
takenAfter: startOfDay.toISOString(),
takenBefore: endOfDay.toISOString()
};
const encodedQuery = encodeURIComponent(JSON.stringify(queryParams));
return `${userSettings.immich_url}/search?query=${encodedQuery}`;
case 'photoprism':
return `${userSettings.photoprism_url}/library/browse?view=cards&year=${photo.localDateTime.split('-')[0]}&month=${photo.localDateTime.split('-')[1]}&order=newest&public=true&quality=3`;
default:
return '#'; // Default or error case
}
}
function getSourceUrl(photo, userSettings) {
switch (photo.source) {
case 'photoprism':
return userSettings.photoprism_url;
case 'immich':
return userSettings.immich_url;
default:
return '#'; // Default or error case
}
}
export function createPhotoMarker(photo, userSettings, photoMarkers, apiKey) {
if (!photo.latitude || !photo.longitude) return;
const thumbnailUrl = `/api/v1/photos/${photo.id}/thumbnail.jpg?api_key=${apiKey}&source=${photo.source}`;
const icon = L.divIcon({
className: 'photo-marker',
@ -229,25 +262,16 @@ export function createPhotoMarker(photo, immichUrl, photoMarkers,apiKey) {
});
const marker = L.marker(
[photo.exifInfo.latitude, photo.exifInfo.longitude],
[photo.latitude, photo.longitude],
{ icon }
);
const startOfDay = new Date(photo.localDateTime);
startOfDay.setHours(0, 0, 0, 0);
const photo_link = getPhotoLink(photo, userSettings);
const source_url = getSourceUrl(photo, userSettings);
const endOfDay = new Date(photo.localDateTime);
endOfDay.setHours(23, 59, 59, 999);
const queryParams = {
takenAfter: startOfDay.toISOString(),
takenBefore: endOfDay.toISOString()
};
const encodedQuery = encodeURIComponent(JSON.stringify(queryParams));
const immich_photo_link = `${immichUrl}/search?query=${encodedQuery}`;
const popupContent = `
<div class="max-w-xs">
<a href="${immich_photo_link}" target="_blank" onmouseover="this.firstElementChild.style.boxShadow = '0 4px 8px rgba(0, 0, 0, 0.3)';"
<a href="${photo_link}" target="_blank" onmouseover="this.firstElementChild.style.boxShadow = '0 4px 8px rgba(0, 0, 0, 0.3)';"
onmouseout="this.firstElementChild.style.boxShadow = '';">
<img src="${thumbnailUrl}"
class="mb-2 rounded"
@ -256,7 +280,8 @@ export function createPhotoMarker(photo, immichUrl, photoMarkers,apiKey) {
</a>
<h3 class="font-bold">${photo.originalFileName}</h3>
<p>Taken: ${new Date(photo.localDateTime).toLocaleString()}</p>
<p>Location: ${photo.exifInfo.city}, ${photo.exifInfo.state}, ${photo.exifInfo.country}</p>
<p>Location: ${photo.city}, ${photo.state}, ${photo.country}</p>
<p>Source: <a href="${source_url}" target="_blank">${photo.source}</a></p>
${photo.type === 'VIDEO' ? '🎥 Video' : '📷 Photo'}
</div>
`;

View file

@ -3,7 +3,7 @@ import { getUrlParameter } from "../maps/helpers";
import { minutesToDaysHoursMinutes } from "../maps/helpers";
import { haversineDistance } from "../maps/helpers";
export function addHighlightOnHover(polyline, map, polylineCoordinates, userSettings) {
export function addHighlightOnHover(polyline, map, polylineCoordinates, userSettings, distanceUnit) {
const originalStyle = { color: "blue", opacity: userSettings.routeOpacity, weight: 3 };
const highlightStyle = { color: "yellow", opacity: 1, weight: 5 };
@ -33,7 +33,7 @@ export function addHighlightOnHover(polyline, map, polylineCoordinates, userSett
<strong>Start:</strong> ${firstTimestamp}<br>
<strong>End:</strong> ${lastTimestamp}<br>
<strong>Duration:</strong> ${timeOnRoute}<br>
<strong>Total Distance:</strong> ${formatDistance(totalDistance, userSettings.distanceUnit)}<br>
<strong>Total Distance:</strong> ${formatDistance(totalDistance, distanceUnit)}<br>
`;
if (isDebugMode) {
@ -90,7 +90,7 @@ export function addHighlightOnHover(polyline, map, polylineCoordinates, userSett
});
}
export function createPolylinesLayer(markers, map, userSettings) {
export function createPolylinesLayer(markers, map, timezone, routeOpacity, userSettings, distanceUnit) {
const splitPolylines = [];
let currentPolyline = [];
const distanceThresholdMeters = parseInt(userSettings.meters_between_routes) || 500;
@ -123,7 +123,7 @@ export function createPolylinesLayer(markers, map, userSettings) {
const latLngs = polylineCoordinates.map((point) => [point[0], point[1]]);
const polyline = L.polyline(latLngs, { color: "blue", opacity: 0.6, weight: 3 });
addHighlightOnHover(polyline, map, polylineCoordinates, userSettings);
addHighlightOnHover(polyline, map, polylineCoordinates, userSettings, distanceUnit);
return polyline;
})

View file

@ -7,7 +7,7 @@ class BulkStatsCalculatingJob < ApplicationJob
user_ids = User.pluck(:id)
user_ids.each do |user_id|
Stats::CalculatingJob.perform_later(user_id)
Stats::BulkCalculator.new(user_id).call
end
end
end

11
app/jobs/cache/preheating_job.rb vendored Normal file
View file

@ -0,0 +1,11 @@
# frozen_string_literal: true
class Cache::PreheatingJob < ApplicationJob
queue_as :default
def perform
User.find_each do |user|
Rails.cache.write("dawarich/user_#{user.id}_years_tracked", user.years_tracked, expires_in: 1.day)
end
end
end

View file

@ -7,6 +7,8 @@ class EnqueueBackgroundJob < ApplicationJob
case job_name
when 'start_immich_import'
Import::ImmichGeodataJob.perform_later(user_id)
when 'start_photoprism_import'
Import::PhotoprismGeodataJob.perform_later(user_id)
when 'start_reverse_geocoding', 'continue_reverse_geocoding'
Jobs::Create.new(job_name, user_id).call
else

View file

@ -0,0 +1,12 @@
# frozen_string_literal: true
class Import::PhotoprismGeodataJob < ApplicationJob
queue_as :imports
sidekiq_options retry: false
def perform(user_id)
user = User.find(user_id)
Photoprism::ImportGeodata.new(user).call
end
end

View file

@ -3,8 +3,8 @@
class Stats::CalculatingJob < ApplicationJob
queue_as :stats
def perform(user_id, start_at: nil, end_at: nil)
Stats::Calculate.new(user_id, start_at:, end_at:).call
def perform(user_id, year, month)
Stats::CalculateMonth.new(user_id, year, month).call
create_stats_updated_notification(user_id)
rescue StandardError => e

View file

@ -0,0 +1,14 @@
# frozen_string_literal: true
class TelemetrySendingJob < ApplicationJob
queue_as :default
def perform
return if ENV['DISABLE_TELEMETRY'] == 'true'
data = Telemetry::Gather.new.call
Rails.logger.info("Telemetry data: #{data}")
Telemetry::Send.new(data).call
end
end

View file

@ -10,10 +10,17 @@ class Import < ApplicationRecord
enum :source, {
google_semantic_history: 0, owntracks: 1, google_records: 2,
google_phone_takeout: 3, gpx: 4, immich_api: 5, geojson: 6
google_phone_takeout: 3, gpx: 4, immich_api: 5, geojson: 6, photoprism_api: 7
}
def process!
Imports::Create.new(user, self).call
end
def years_and_months_tracked
points.order(:timestamp).pluck(:timestamp).map do |timestamp|
time = Time.zone.at(timestamp)
[time.year, time.month]
end.uniq
end
end

View file

@ -6,39 +6,16 @@ class Stat < ApplicationRecord
belongs_to :user
def distance_by_day
timespan.to_a.map.with_index(1) do |day, index|
beginning_of_day = day.beginning_of_day.to_i
end_of_day = day.end_of_day.to_i
# We have to filter by user as well
points = user
.tracked_points
.without_raw_data
.order(timestamp: :asc)
.where(timestamp: beginning_of_day..end_of_day)
data = { day: index, distance: 0 }
points.each_cons(2) do |point1, point2|
distance = Geocoder::Calculations.distance_between(
point1.to_coordinates, point2.to_coordinates, units: ::DISTANCE_UNIT
)
data[:distance] += distance
end
[data[:day], data[:distance].round(2)]
end
monthly_points = points
calculate_daily_distances(monthly_points)
end
def self.year_distance(year, user)
stats = where(year:, user:).order(:month)
(1..12).to_a.map do |month|
month_stat = stats.select { |stat| stat.month == month }.first
stats_by_month = where(year:, user:).order(:month).index_by(&:month)
(1..12).map do |month|
month_name = Date::MONTHNAMES[month]
distance = month_stat&.distance || 0
distance = stats_by_month[month]&.distance || 0
[month_name, distance]
end
@ -58,10 +35,11 @@ class Stat < ApplicationRecord
}
end
def self.years
starting_year = select(:year).min&.year || Time.current.year
(starting_year..Time.current.year).to_a.reverse
def points
user.tracked_points
.without_raw_data
.where(timestamp: timespan)
.order(timestamp: :asc)
end
private
@ -69,4 +47,25 @@ class Stat < ApplicationRecord
def timespan
DateTime.new(year, month).beginning_of_month..DateTime.new(year, month).end_of_month
end
def calculate_daily_distances(monthly_points)
timespan.to_a.map.with_index(1) do |day, index|
daily_points = filter_points_for_day(monthly_points, day)
distance = calculate_distance(daily_points)
[index, distance.round(2)]
end
end
def filter_points_for_day(points, day)
beginning_of_day = day.beginning_of_day.to_i
end_of_day = day.end_of_day.to_i
points.select { |p| p.timestamp.between?(beginning_of_day, end_of_day) }
end
def calculate_distance(points)
points.each_cons(2).sum do |point1, point2|
DistanceCalculator.new(point1, point2).call
end
end
end

View file

@ -17,39 +17,34 @@ class Trip < ApplicationRecord
points.pluck(:country).uniq.compact
end
def photos
return [] if user.settings['immich_url'].blank? || user.settings['immich_api_key'].blank?
def photo_previews
@photo_previews ||= select_dominant_orientation(photos).sample(12)
end
immich_photos = Immich::RequestPhotos.new(
user,
start_date: started_at.to_date.to_s,
end_date: ended_at.to_date.to_s
).call.reject { |asset| asset['type'].downcase == 'video' }
# let's count what photos are more: vertical or horizontal and select the ones that are more
vertical_photos = immich_photos.select { _1['exifInfo']['orientation'] == '6' }
horizontal_photos = immich_photos.select { _1['exifInfo']['orientation'] == '3' }
# this is ridiculous, but I couldn't find my way around frontend
# to show all photos in the same height
photos = vertical_photos.count > horizontal_photos.count ? vertical_photos : horizontal_photos
photos.sample(12).sort_by { _1['localDateTime'] }.map do |asset|
{ url: "/api/v1/photos/#{asset['id']}/thumbnail.jpg?api_key=#{user.api_key}" }
end
def photo_sources
@photo_sources ||= photos.map { _1[:source] }.uniq
end
private
def photos
@photos ||= Trips::Photos.new(self, user).call
end
def select_dominant_orientation(photos)
vertical_photos = photos.select { |photo| photo[:orientation] == 'portrait' }
horizontal_photos = photos.select { |photo| photo[:orientation] == 'landscape' }
# this is ridiculous, but I couldn't find my way around frontend
# to show all photos in the same height
vertical_photos.count > horizontal_photos.count ? vertical_photos : horizontal_photos
end
def calculate_distance
distance = 0
points.each_cons(2) do |point1, point2|
distance_between = Geocoder::Calculations.distance_between(
point1.to_coordinates, point2.to_coordinates, units: ::DISTANCE_UNIT
)
distance += distance_between
distance += DistanceCalculator.new(point1, point2).call
end
self.distance = distance.round

View file

@ -2,9 +2,9 @@
class User < ApplicationRecord
# Include default devise modules. Others available are:
# :confirmable, :lockable, :timeoutable, :trackable and :omniauthable
# :confirmable, :lockable, :timeoutable, and :omniauthable
devise :database_authenticatable, :registerable,
:recoverable, :rememberable, :validatable
:recoverable, :rememberable, :validatable, :trackable
has_many :tracked_points, class_name: 'Point', dependent: :destroy
has_many :imports, dependent: :destroy
@ -18,6 +18,7 @@ class User < ApplicationRecord
has_many :trips, dependent: :destroy
after_create :create_api_key
before_save :strip_trailing_slashes
def countries_visited
stats.pluck(:toponyms).flatten.map { _1['country'] }.uniq.compact
@ -57,6 +58,25 @@ class User < ApplicationRecord
tracked_points.where(geodata: {}).count
end
def immich_integration_configured?
settings['immich_url'].present? && settings['immich_api_key'].present?
end
def photoprism_integration_configured?
settings['photoprism_url'].present? && settings['photoprism_api_key'].present?
end
def years_tracked
Rails.cache.fetch("dawarich/user_#{id}_years_tracked", expires_in: 1.day) do
tracked_points
.pluck(:timestamp)
.map { |ts| Time.zone.at(ts).year }
.uniq
.sort
.reverse
end
end
private
def create_api_key
@ -64,4 +84,9 @@ class User < ApplicationRecord
save
end
def strip_trailing_slashes
settings['immich_url']&.gsub!(%r{/+\z}, '')
settings['photoprism_url']&.gsub!(%r{/+\z}, '')
end
end

View file

@ -28,7 +28,9 @@ class Visit < ApplicationRecord
def default_radius
return area&.radius if area.present?
radius = points.map { Geocoder::Calculations.distance_between(center, [_1.latitude, _1.longitude]) }.max
radius = points.map do |point|
DistanceCalculator.new(center, [point.latitude, point.longitude]).call
end.max
radius && radius >= 15 ? radius : 15
end

View file

@ -0,0 +1,73 @@
# frozen_string_literal: true
class Api::PhotoSerializer
def initialize(photo, source)
@photo = photo.with_indifferent_access
@source = source
end
def call
{
id: id,
latitude: latitude,
longitude: longitude,
localDateTime: local_date_time,
originalFileName: original_file_name,
city: city,
state: state,
country: country,
type: type,
orientation: orientation,
source: source
}
end
private
attr_reader :photo, :source
def id
photo['id'] || photo['Hash']
end
def latitude
photo.dig('exifInfo', 'latitude') || photo['Lat']
end
def longitude
photo.dig('exifInfo', 'longitude') || photo['Lng']
end
def local_date_time
photo['localDateTime'] || photo['TakenAtLocal']
end
def original_file_name
photo['originalFileName'] || photo['OriginalName']
end
def city
photo.dig('exifInfo', 'city') || photo['PlaceCity']
end
def state
photo.dig('exifInfo', 'state') || photo['PlaceState']
end
def country
photo.dig('exifInfo', 'country') || photo['PlaceCountry']
end
def type
(photo['type'] || photo['Type']).downcase
end
def orientation
case source
when 'immich'
photo.dig('exifInfo', 'orientation') == '6' ? 'portrait' : 'landscape'
when 'photoprism'
photo['Portrait'] ? 'portrait' : 'landscape'
end
end
end

View file

@ -0,0 +1,18 @@
# frozen_string_literal: true
class DistanceCalculator
def initialize(point1, point2)
@point1 = point1
@point2 = point2
end
def call
Geocoder::Calculations.distance_between(
point1.to_coordinates, point2.to_coordinates, units: ::DISTANCE_UNIT
)
end
private
attr_reader :point1, :point2
end

View file

@ -57,7 +57,7 @@ class Immich::ImportGeodata
end
def log_no_data
Rails.logger.info 'No data found'
Rails.logger.info 'No geodata found for Immich'
end
def create_import_failed_notification(import_name)

View file

@ -5,7 +5,7 @@ class Immich::RequestPhotos
def initialize(user, start_date: '1970-01-01', end_date: nil)
@user = user
@immich_api_base_url = "#{user.settings['immich_url']}/api/search/metadata"
@immich_api_base_url = URI.parse("#{user.settings['immich_url']}/api/search/metadata")
@immich_api_key = user.settings['immich_api_key']
@start_date = start_date
@end_date = end_date
@ -37,15 +37,7 @@ class Immich::RequestPhotos
items = response.dig('assets', 'items')
if items.blank?
Rails.logger.debug('==== IMMICH RESPONSE WITH NO ITEMS ====')
Rails.logger.debug("START_DATE: #{start_date}")
Rails.logger.debug("END_DATE: #{end_date}")
Rails.logger.debug(response)
Rails.logger.debug('==== IMMICH RESPONSE WITH NO ITEMS ====')
break
end
break if items.blank?
data << items

View file

@ -24,20 +24,19 @@ class Imports::Create
def parser(source)
# Bad classes naming by the way, they are not parsers, they are point creators
case source
when 'google_semantic_history' then GoogleMaps::SemanticHistoryParser
when 'google_phone_takeout' then GoogleMaps::PhoneTakeoutParser
when 'owntracks' then OwnTracks::ExportParser
when 'gpx' then Gpx::TrackParser
when 'immich_api' then Immich::ImportParser
when 'geojson' then Geojson::ImportParser
when 'google_semantic_history' then GoogleMaps::SemanticHistoryParser
when 'google_phone_takeout' then GoogleMaps::PhoneTakeoutParser
when 'owntracks' then OwnTracks::ExportParser
when 'gpx' then Gpx::TrackParser
when 'geojson' then Geojson::ImportParser
when 'immich_api', 'photoprism_api' then Photos::ImportParser
end
end
def schedule_stats_creating(user_id)
start_at = import.points.order(:timestamp).first.recorded_at
end_at = import.points.order(:timestamp).last.recorded_at
Stats::CalculatingJob.perform_later(user_id, start_at:, end_at:)
import.years_and_months_tracked.each do |year, month|
Stats::CalculatingJob.perform_later(user_id, year, month)
end
end
def schedule_visit_suggesting(user_id, import)

View file

@ -21,6 +21,6 @@ class Jobs::Create
raise InvalidJobName, 'Invalid job name'
end
points.each(&:async_reverse_geocode)
points.find_each(batch_size: 1_000, &:async_reverse_geocode)
end
end

View file

@ -9,7 +9,12 @@ class OwnTracks::RecParser
def call
file.split("\n").map do |line|
JSON.parse(line.split("\t* \t")[1])
end
parts = line.split("\t")
if parts.size > 2 && parts[1].strip == '*'
JSON.parse(parts[2])
else
nil
end
end.compact
end
end

View file

@ -0,0 +1,16 @@
# frozen_string_literal: true
class Photoprism::CachePreviewToken
attr_reader :user, :preview_token
TOKEN_CACHE_KEY = 'dawarich/photoprism_preview_token'
def initialize(user, preview_token)
@user = user
@preview_token = preview_token
end
def call
Rails.cache.write("#{TOKEN_CACHE_KEY}_#{user.id}", preview_token)
end
end

View file

@ -0,0 +1,86 @@
# frozen_string_literal: true
class Photoprism::ImportGeodata
attr_reader :user, :start_date, :end_date
def initialize(user, start_date: '1970-01-01', end_date: nil)
@user = user
@start_date = start_date
@end_date = end_date
end
def call
photoprism_data = retrieve_photoprism_data
return log_no_data if photoprism_data.empty?
json_data = parse_photoprism_data(photoprism_data)
create_and_process_import(json_data)
end
private
def create_and_process_import(json_data)
import = find_or_create_import(json_data)
return create_import_failed_notification(import.name) unless import.new_record?
import.update!(raw_data: json_data)
ImportJob.perform_later(user.id, import.id)
end
def find_or_create_import(json_data)
user.imports.find_or_initialize_by(
name: file_name(json_data),
source: :photoprism_api
)
end
def retrieve_photoprism_data
Photoprism::RequestPhotos.new(user, start_date:, end_date:).call
end
def parse_photoprism_data(photoprism_data)
geodata = photoprism_data.map do |asset|
next unless valid?(asset)
extract_geodata(asset)
end
geodata.compact.sort_by { |data| data[:timestamp] }
end
def valid?(asset)
asset['Lat'] &&
asset['Lat'] != 0 &&
asset['Lng'] &&
asset['Lng'] != 0 &&
asset['TakenAt']
end
def extract_geodata(asset)
{
latitude: asset['Lat'],
longitude: asset['Lng'],
timestamp: Time.zone.parse(asset['TakenAt']).to_i
}
end
def log_no_data
Rails.logger.info 'No geodata found for Photoprism'
end
def create_import_failed_notification(import_name)
Notifications::Create.new(
user:,
kind: :info,
title: 'Import was not created',
content: "Import with the same name (#{import_name}) already exists. If you want to proceed, delete the existing import and try again."
).call
end
def file_name(photoprism_data_json)
from = Time.zone.at(photoprism_data_json.first[:timestamp]).to_date
to = Time.zone.at(photoprism_data_json.last[:timestamp]).to_date
"photoprism-geodata-#{user.email}-from-#{from}-to-#{to}.json"
end
end

View file

@ -0,0 +1,101 @@
# frozen_string_literal: true
# This integration built based on
# [September 15, 2024](https://github.com/photoprism/photoprism/releases/tag/240915-e1280b2fb)
# release of Photoprism.
class Photoprism::RequestPhotos
attr_reader :user, :photoprism_api_base_url, :photoprism_api_key, :start_date, :end_date
def initialize(user, start_date: '1970-01-01', end_date: nil)
@user = user
@photoprism_api_base_url = URI.parse("#{user.settings['photoprism_url']}/api/v1/photos")
@photoprism_api_key = user.settings['photoprism_api_key']
@start_date = start_date
@end_date = end_date
end
def call
raise ArgumentError, 'Photoprism URL is missing' if user.settings['photoprism_url'].blank?
raise ArgumentError, 'Photoprism API key is missing' if photoprism_api_key.blank?
data = retrieve_photoprism_data
return [] if data.blank? || data[0]['error'].present?
time_framed_data(data, start_date, end_date)
end
private
def retrieve_photoprism_data
data = []
offset = 0
while offset < 1_000_000
response_data = fetch_page(offset)
break if response_data.blank? || (response_data.is_a?(Hash) && response_data.try(:[], 'error').present?)
data << response_data
offset += 1000
end
data.flatten
end
def fetch_page(offset)
response = HTTParty.get(
photoprism_api_base_url,
headers: headers,
query: request_params(offset)
)
if response.code != 200
Rails.logger.error "Photoprism API returned #{response.code}: #{response.body}"
Rails.logger.debug "Photoprism API request params: #{request_params(offset).inspect}"
end
cache_preview_token(response.headers)
JSON.parse(response.body)
end
def headers
{
'Authorization' => "Bearer #{photoprism_api_key}",
'accept' => 'application/json'
}
end
def request_params(offset = 0)
params = offset.zero? ? default_params : default_params.merge(offset: offset)
params[:before] = end_date if end_date.present?
params
end
def default_params
{
q: '',
public: true,
quality: 3,
after: start_date,
count: 1000
}
end
def time_framed_data(data, start_date, end_date)
data.flatten.select do |photo|
taken_at = DateTime.parse(photo['TakenAtLocal'])
end_date ||= Time.current
taken_at.between?(start_date.to_datetime, end_date.to_datetime)
end
end
def cache_preview_token(headers)
preview_token = headers['X-Preview-Token']
Photoprism::CachePreviewToken.new(user, preview_token).call
end
end

View file

@ -1,6 +1,6 @@
# frozen_string_literal: true
class Immich::ImportParser
class Photos::ImportParser
include Imports::Broadcaster
attr_reader :import, :json, :user_id

View file

@ -0,0 +1,45 @@
# frozen_string_literal: true
class Photos::Search
attr_reader :user, :start_date, :end_date
def initialize(user, start_date: '1970-01-01', end_date: nil)
@user = user
@start_date = start_date
@end_date = end_date
end
def call
photos = []
photos << request_immich if user.immich_integration_configured?
photos << request_photoprism if user.photoprism_integration_configured?
photos.flatten.map { |photo| Api::PhotoSerializer.new(photo, photo[:source]).call }
end
private
def request_immich
Immich::RequestPhotos.new(
user,
start_date: start_date,
end_date: end_date
).call.map { |asset| transform_asset(asset, 'immich') }.compact
end
def request_photoprism
Photoprism::RequestPhotos.new(
user,
start_date: start_date,
end_date: end_date
).call.map { |asset| transform_asset(asset, 'photoprism') }.compact
end
def transform_asset(asset, source)
asset_type = asset['type'] || asset['Type']
return if asset_type.downcase == 'video'
asset.merge(source: source)
end
end

View file

@ -0,0 +1,51 @@
# frozen_string_literal: true
class Photos::Thumbnail
def initialize(user, source, id)
@user = user
@source = source
@id = id
end
def call
HTTParty.get(request_url, headers: headers)
end
private
attr_reader :user, :source, :id
def source_url
user.settings["#{source}_url"]
end
def source_api_key
user.settings["#{source}_api_key"]
end
def source_path
case source
when 'immich'
"/api/assets/#{id}/thumbnail?size=preview"
when 'photoprism'
preview_token = Rails.cache.read("#{Photoprism::CachePreviewToken::TOKEN_CACHE_KEY}_#{user.id}")
"/api/v1/t/#{id}/#{preview_token}/tile_500"
else
raise "Unsupported source: #{source}"
end
end
def request_url
"#{source_url}#{source_path}"
end
def headers
request_headers = {
'accept' => 'application/octet-stream'
}
request_headers['X-Api-Key'] = source_api_key if source == 'immich'
request_headers
end
end

View file

@ -0,0 +1,40 @@
# frozen_string_literal: true
module Stats
class BulkCalculator
def initialize(user_id)
@user_id = user_id
end
def call
months = extract_months(fetch_timestamps)
schedule_calculations(months)
end
private
attr_reader :user_id
def fetch_timestamps
last_calculated_at = Stat.where(user_id:).maximum(:updated_at)
last_calculated_at ||= DateTime.new(1970, 1, 1)
time_diff = last_calculated_at.to_i..Time.current.to_i
Point.where(user_id:, timestamp: time_diff).pluck(:timestamp)
end
def extract_months(timestamps)
timestamps.group_by do |timestamp|
time = Time.zone.at(timestamp)
[time.year, time.month]
end.keys
end
def schedule_calculations(months)
months.each do |year, month|
Stats::CalculatingJob.perform_later(user_id, year, month)
end
end
end
end

View file

@ -1,69 +0,0 @@
# frozen_string_literal: true
class Stats::Calculate
def initialize(user_id, start_at: nil, end_at: nil)
@user = User.find(user_id)
@start_at = start_at || DateTime.new(1970, 1, 1)
@end_at = end_at || Time.current
end
def call
points = points(start_timestamp, end_timestamp)
points_by_month = points.group_by_month(&:recorded_at)
points_by_month.each do |month, month_points|
update_month_stats(month_points, month.year, month.month)
end
rescue StandardError => e
create_stats_update_failed_notification(user, e)
end
private
attr_reader :user, :start_at, :end_at
def start_timestamp = start_at.to_i
def end_timestamp = end_at.to_i
def update_month_stats(month_points, year, month)
return if month_points.empty?
stat = current_stat(year, month)
distance_by_day = stat.distance_by_day
stat.daily_distance = distance_by_day
stat.distance = distance(distance_by_day)
stat.toponyms = toponyms(month_points)
stat.save
end
def points(start_at, end_at)
user
.tracked_points
.without_raw_data
.where(timestamp: start_at..end_at)
.order(:timestamp)
.select(:latitude, :longitude, :timestamp, :city, :country)
end
def distance(distance_by_day)
distance_by_day.sum { |day| day[1] }
end
def toponyms(points)
CountriesAndCities.new(points).call
end
def current_stat(year, month)
Stat.find_or_initialize_by(year:, month:, user:)
end
def create_stats_update_failed_notification(user, error)
Notifications::Create.new(
user:,
kind: :error,
title: 'Stats update failed',
content: "#{error.message}, stacktrace: #{error.backtrace.join("\n")}"
).call
end
end

View file

@ -0,0 +1,69 @@
# frozen_string_literal: true
class Stats::CalculateMonth
def initialize(user_id, year, month)
@user = User.find(user_id)
@year = year
@month = month
end
def call
return if points.empty?
update_month_stats(year, month)
rescue StandardError => e
create_stats_update_failed_notification(user, e)
end
private
attr_reader :user, :year, :month
def start_timestamp = DateTime.new(year, month, 1).to_i
def end_timestamp
DateTime.new(year, month, -1).to_i # -1 returns last day of month
end
def update_month_stats(year, month)
Stat.transaction do
stat = Stat.find_or_initialize_by(year:, month:, user:)
distance_by_day = stat.distance_by_day
stat.assign_attributes(
daily_distance: distance_by_day,
distance: distance(distance_by_day),
toponyms: toponyms
)
stat.save
end
end
def points
return @points if defined?(@points)
@points = user
.tracked_points
.without_raw_data
.where(timestamp: start_timestamp..end_timestamp)
.select(:latitude, :longitude, :timestamp, :city, :country)
.order(timestamp: :asc)
end
def distance(distance_by_day)
distance_by_day.sum { |day| day[1] }
end
def toponyms
CountriesAndCities.new(points).call
end
def create_stats_update_failed_notification(user, error)
Notifications::Create.new(
user:,
kind: :error,
title: 'Stats update failed',
content: "#{error.message}, stacktrace: #{error.backtrace.join("\n")}"
).call
end
end

View file

@ -0,0 +1,32 @@
# frozen_string_literal: true
class Telemetry::Gather
def initialize(measurement: 'dawarich_usage_metrics')
@measurement = measurement
end
def call
{
measurement:,
timestamp: Time.current.to_i,
tags: { instance_id: },
fields: { dau:, app_version: }
}
end
private
attr_reader :measurement
def instance_id
@instance_id ||= Digest::SHA2.hexdigest(User.first.api_key)
end
def app_version
"\"#{APP_VERSION}\""
end
def dau
User.where(last_sign_in_at: Time.zone.today.beginning_of_day..Time.zone.today.end_of_day).count
end
end

View file

@ -0,0 +1,46 @@
# frozen_string_literal: true
class Telemetry::Send
BUCKET = 'dawarich_metrics'
ORG = 'monitoring'
def initialize(payload)
@payload = payload
end
def call
return if ENV['DISABLE_TELEMETRY'] == 'true'
line_protocol = build_line_protocol
response = send_request(line_protocol)
handle_response(response)
end
private
attr_reader :payload
def build_line_protocol
tag_string = payload[:tags].map { |k, v| "#{k}=#{v}" }.join(',')
field_string = payload[:fields].map { |k, v| "#{k}=#{v}" }.join(',')
"#{payload[:measurement]},#{tag_string} #{field_string} #{payload[:timestamp].to_i}"
end
def send_request(line_protocol)
HTTParty.post(
"#{TELEMETRY_URL}?org=#{ORG}&bucket=#{BUCKET}&precision=s",
body: line_protocol,
headers: {
'Authorization' => "Token #{Base64.decode64(TELEMETRY_STRING)}",
'Content-Type' => 'text/plain'
}
)
end
def handle_response(response)
Rails.logger.error("InfluxDB write failed: #{response.body}") unless response.success?
response
end
end

View file

@ -0,0 +1,43 @@
# frozen_string_literal: true
class Trips::Photos
def initialize(trip, user)
@trip = trip
@user = user
end
def call
return [] unless can_fetch_photos?
photos
end
private
attr_reader :trip, :user
def can_fetch_photos?
user.immich_integration_configured? || user.photoprism_integration_configured?
end
def photos
return @photos if defined?(@photos)
photos = Photos::Search.new(
user,
start_date: trip.started_at.to_date.to_s,
end_date: trip.ended_at.to_date.to_s
).call
@photos = photos.map { |photo| photo_thumbnail(photo) }
end
def photo_thumbnail(asset)
{
id: asset[:id],
url: "/api/v1/photos/#{asset[:id]}/thumbnail.jpg?api_key=#{user.api_key}&source=#{asset[:source]}",
source: asset[:source],
orientation: asset[:orientation]
}
end
end

View file

@ -10,6 +10,11 @@
<% else %>
<a href='' class="rounded-lg py-3 px-5 bg-blue-900 text-gray block font-medium tooltip cursor-not-allowed" data-tip="You need to provide your Immich instance data in the Settings">Import Immich data</a>
<% end %>
<% if current_user.settings['photoprism_url'] && current_user.settings['photoprism_api_key'] %>
<%= link_to 'Import Photoprism data', settings_background_jobs_path(job_name: 'start_photoprism_import'), method: :post, data: { confirm: 'Are you sure?', turbo_confirm: 'Are you sure?', turbo_method: :post }, class: 'rounded-lg py-3 px-5 bg-blue-600 text-white block font-medium' %>
<% else %>
<a href='' class="rounded-lg py-3 px-5 bg-blue-900 text-gray block font-medium tooltip cursor-not-allowed" data-tip="You need to provide your Photoprism instance data in the Settings">Import Photoprism data</a>
<% end %>
</div>
<div id="imports" class="min-w-full">

View file

@ -44,7 +44,7 @@
<div
id='map'
class="w-full"
class="w-full z-0"
data-controller="maps points"
data-points-target="map"
data-distance_unit="<%= DISTANCE_UNIT %>"

View file

@ -1,5 +1,5 @@
<div role="tablist" class="tabs tabs-lifted tabs-lg">
<%= link_to 'Main', settings_path, role: 'tab', class: "tab #{active_tab?(settings_path)}" %>
<%= link_to 'Integrations', settings_path, role: 'tab', class: "tab #{active_tab?(settings_path)}" %>
<% if current_user.admin? %>
<%= link_to 'Users', settings_users_path, role: 'tab', class: "tab #{active_tab?(settings_users_path)}" %>
<%= link_to 'Background Jobs', settings_background_jobs_path, role: 'tab', class: "tab #{active_tab?(settings_background_jobs_path)}" %>

View file

@ -5,7 +5,7 @@
<div class="flex flex-col lg:flex-row w-full my-10 space-x-4">
<div class="card flex-shrink-0 w-full max-w-sm shadow-2xl bg-base-100 px-5 py-5 mx-5">
<h2 class="text-2xl font-bold">Edit your Dawarich settings!</h1>
<h2 class="text-2xl font-bold">Edit your Integrations settings!</h1>
<%= form_for :settings, url: settings_path, method: :patch, data: { turbo_method: :patch, turbo: false } do |f| %>
<div class="form-control my-2">
<%= f.label :immich_url %>
@ -15,6 +15,16 @@
<%= f.label :immich_api_key %>
<%= f.text_field :immich_api_key, value: current_user.settings['immich_api_key'], class: "input input-bordered", placeholder: 'xxxxxxxxxxxxxx' %>
</div>
<div class="divider"></div>
<div class="form-control my-2">
<%= f.label :photoprism_url %>
<%= f.text_field :photoprism_url, value: current_user.settings['photoprism_url'], class: "input input-bordered", placeholder: 'http://192.168.0.1:2342' %>
</div>
<div class="form-control my-2">
<%= f.label :photoprism_api_key %>
<%= f.text_field :photoprism_api_key, value: current_user.settings['photoprism_api_key'], class: "input input-bordered", placeholder: 'xxxxxxxxxxxxxx' %>
</div>
<div class="form-control my-2">
<%= f.submit "Update", class: "btn btn-primary" %>
</div>

View file

@ -4,7 +4,7 @@
<div class="dropdown">
<div tabindex="0" role="button" class="btn">Select year</div>
<ul tabindex="0" class="dropdown-content z-[1] menu p-2 shadow bg-base-100 rounded-box w-52">
<% current_user.stats.years.each do |year| %>
<% current_user.years_tracked.each do |year| %>
<li><%= link_to year, map_url(year_timespan(year).merge(year: year, import_id: params[:import_id])) %></li>
<% end %>
</ul>

View file

@ -10,7 +10,7 @@
<div style="width: 100%; aspect-ratio: 1/1;"
id="map-<%= trip.id %>"
class="rounded-lg"
class="rounded-lg z-0"
data-controller="trip-map"
data-trip-map-trip-id-value="<%= trip.id %>"
data-trip-map-coordinates-value="<%= trip.points.pluck(:latitude, :longitude, :battery, :altitude, :timestamp, :velocity, :id, :country).to_json %>"

View file

@ -18,7 +18,7 @@
<div class="w-full">
<div
id='map'
class="w-full h-full rounded-lg"
class="w-full h-full rounded-lg z-0"
data-controller="trips"
data-trips-target="container"
data-distance_unit="<%= DISTANCE_UNIT %>"
@ -36,8 +36,8 @@
</div>
<!-- Photos Grid Section -->
<% if @photos.any? %>
<% @photos.each_slice(4) do |slice| %>
<% if @photo_previews.any? %>
<% @photo_previews.each_slice(4) do |slice| %>
<div class="h-32 flex gap-4 mt-4 justify-center">
<% slice.each do |photo| %>
<div class="flex-1 h-full overflow-hidden rounded-lg transition-transform duration-300 hover:scale-105 hover:shadow-lg">
@ -52,9 +52,13 @@
<% end %>
<% end %>
<div class="text-center mt-6">
<%= link_to "More photos on Immich", immich_search_url(current_user.settings['immich_url'], @trip.started_at, @trip.ended_at), class: "btn btn-primary", target: '_blank' %>
</div>
<% if @photo_sources.any? %>
<div class="text-center mt-6">
<% @photo_sources.each do |source| %>
<%= link_to "More photos on #{source}", photo_search_url(source, current_user.settings, @trip.started_at, @trip.ended_at), class: "btn btn-primary mt-2", target: '_blank' %>
<% end %>
</div>
<% end %>
</div>
</div>
</div>

View file

@ -6,3 +6,5 @@ PHOTON_API_HOST = ENV.fetch('PHOTON_API_HOST', nil)
PHOTON_API_USE_HTTPS = ENV.fetch('PHOTON_API_USE_HTTPS', 'true') == 'true'
DISTANCE_UNIT = ENV.fetch('DISTANCE_UNIT', 'km').to_sym
APP_VERSION = File.read('.app_version').strip
TELEMETRY_STRING = Base64.encode64('IjVFvb8j3P9-ArqhSGav9j8YcJaQiuNIzkfOPKQDk2lvKXqb8t1NSRv50oBkaKtlrB_ZRzO9NdurpMtncV_HYQ==')
TELEMETRY_URL = 'https://influxdb2.frey.today/api/v2/write'

View file

@ -0,0 +1,3 @@
# frozen_string_literal: true
Rails.cache.delete('dawarich/app-version-check')

View file

@ -7,7 +7,7 @@ if !Rails.env.test? && ENV['PROMETHEUS_EXPORTER_ENABLED'].to_s == 'true'
# This reports stats per request like HTTP status and timings
Rails.application.middleware.unshift PrometheusExporter::Middleware
# this reports basic process stats like RSS and GC info
# This reports basic process stats like RSS and GC info
PrometheusExporter::Instrumentation::Process.start(type: 'web')
# Add ActiveRecord instrumentation

View file

@ -6,14 +6,16 @@ Sidekiq.configure_server do |config|
if ENV['PROMETHEUS_EXPORTER_ENABLED'].to_s == 'true'
require 'prometheus_exporter/instrumentation'
# Add middleware for collecting job-level metrics
config.server_middleware do |chain|
chain.add PrometheusExporter::Instrumentation::Sidekiq
end
# Capture metrics for failed jobs
config.death_handlers << PrometheusExporter::Instrumentation::Sidekiq.death_handler
# Start Prometheus instrumentation
config.on :startup do
PrometheusExporter::Instrumentation::Process.start type: 'sidekiq'
PrometheusExporter::Instrumentation::SidekiqProcess.start
PrometheusExporter::Instrumentation::SidekiqQueue.start
PrometheusExporter::Instrumentation::SidekiqStats.start
@ -25,4 +27,4 @@ Sidekiq.configure_client do |config|
config.redis = { url: ENV['REDIS_URL'] }
end
Sidekiq::Queue['reverse_geocoding'].limit = 1 if PHOTON_API_HOST == 'photon.komoot.io'
Sidekiq::Queue['reverse_geocoding'].limit = 1 if Sidekiq.server? && PHOTON_API_HOST == 'photon.komoot.io'

View file

@ -1,7 +1,7 @@
# config/schedule.yml
bulk_stats_calculating_job:
cron: "0 */6 * * *" # every 6 hour
cron: "0 */1 * * *" # every 1 hour
class: "BulkStatsCalculatingJob"
queue: stats
@ -25,3 +25,13 @@ app_version_checking_job:
cron: "0 */6 * * *" # every 6 hours
class: "AppVersionCheckingJob"
queue: default
telemetry_sending_job:
cron: "0 */1 * * *" # every 1 hour
class: "TelemetrySendingJob"
queue: default
cache_preheating_job:
cron: "0 0 * * *" # every day at 0:00
class: "Cache::PreheatingJob"
queue: default

View file

@ -0,0 +1,43 @@
# frozen_string_literal: true
class CreateTelemetryNotification < ActiveRecord::Migration[7.2]
def up
User.find_each do |user|
Notifications::Create.new(
user:, kind: :info, title: 'Telemetry enabled', content: notification_content
).call
end
end
def down
raise ActiveRecord::IrreversibleMigration
end
private
def notification_content
<<~CONTENT
<p>With the release 0.19.2, Dawarich now can collect usage some metrics and send them to InfluxDB.</p>
<br>
<p>Before this release, the only metrics that could be somehow tracked by developers (only <a href="https://github.com/Freika" class="underline">Freika</a>, as of now) were the number of stars on GitHub and the overall number of docker images being pulled, across all versions of Dawarich, non-splittable by version. New in-app telemetry will allow us to track more granular metrics, allowing me to make decisions based on facts, not just guesses.</p>
<br>
<p>I'm aware about the privacy concerns, so I want to be very transparent about what data is being sent and how it's used.</p>
<br>
<p>Data being sent:</p>
<br>
<ul class="list-disc">
<li>Number of DAU (Daily Active Users)</li>
<li>App version</li>
<li>Instance ID (unique identifier of the Dawarich instance built by hashing the api key of the first user in the database)</li>
</ul>
<br>
<p>The data is being sent to a InfluxDB instance hosted by me and won't be shared with anyone.</p>
<br>
<p>Basically this set of metrics allows me to see how many people are using Dawarich and what versions they are using. No other data is being sent, nor it gives me any knowledge about individual users or their data or activity.</p>
<br>
<p>The telemetry is enabled by default, but it <strong class="text-info underline">can be disabled</strong> by setting <code>DISABLE_TELEMETRY</code> env var to <code>true</code>. The dataset might change in the future, but any changes will be documented here in the changelog and in every release as well as on the <a href="https://dawarich.app/docs/tutorials/telemetry" class="underline">telemetry page</a> of the website docs.</p>
<br>
<p>You can read more about it in the <a href="https://github.com/Freika/dawarich/releases/tag/0.19.2" class="underline">release page</a>.</p>
CONTENT
end
end

View file

@ -0,0 +1,13 @@
# frozen_string_literal: true
class AddDeviseTrackableColumnsToUsers < ActiveRecord::Migration[7.2]
def change
change_table :users, bulk: true do |t|
t.integer :sign_in_count, default: 0, null: false
t.datetime :current_sign_in_at
t.datetime :last_sign_in_at
t.string :current_sign_in_ip
t.string :last_sign_in_ip
end
end
end

View file

@ -0,0 +1,9 @@
# frozen_string_literal: true
class AddIndexToPointsTimestamp < ActiveRecord::Migration[7.2]
disable_ddl_transaction!
def change
add_index :points, %i[user_id timestamp], algorithm: :concurrently
end
end

7
db/schema.rb generated
View file

@ -10,7 +10,7 @@
#
# It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema[7.2].define(version: 2024_12_02_114820) do
ActiveRecord::Schema[7.2].define(version: 2024_12_05_160055) do
# These are extensions that must be enabled in order to support this database
enable_extension "plpgsql"
@ -210,6 +210,11 @@ ActiveRecord::Schema[7.2].define(version: 2024_12_02_114820) do
t.string "theme", default: "dark", null: false
t.jsonb "settings", default: {"fog_of_war_meters"=>"100", "meters_between_routes"=>"1000", "minutes_between_routes"=>"60"}
t.boolean "admin", default: false
t.integer "sign_in_count", default: 0, null: false
t.datetime "current_sign_in_at"
t.datetime "last_sign_in_at"
t.string "current_sign_in_ip"
t.string "last_sign_in_ip"
t.index ["email"], name: "index_users_on_email", unique: true
t.index ["reset_password_token"], name: "index_users_on_reset_password_token", unique: true
end

View file

@ -69,6 +69,7 @@ services:
PROMETHEUS_EXPORTER_ENABLED: false
PROMETHEUS_EXPORTER_HOST: 0.0.0.0
PROMETHEUS_EXPORTER_PORT: 9394
DISABLE_TELEMETRY: false # More on telemetry: https://dawarich.app/docs/tutorials/telemetry
logging:
driver: "json-file"
options:
@ -123,6 +124,7 @@ services:
PROMETHEUS_EXPORTER_ENABLED: false
PROMETHEUS_EXPORTER_HOST: dawarich_app
PROMETHEUS_EXPORTER_PORT: 9394
DISABLE_TELEMETRY: false # More on telemetry: https://dawarich.app/docs/tutorials/telemetry
logging:
driver: "json-file"
options:

View file

@ -8,4 +8,4 @@ This command use [docker-compose.yml](../docker-compose.yml) to build your local
When this command done successfully and all services in containers will start you can open Dawarich web UI by this link [http://127.0.0.1:3000](http://127.0.0.1:3000).
Default credentials for first login in are `user@domain.com` and `password`.
Default credentials for first login in are `demo@dawarich.app` and `password`.

View file

@ -23,7 +23,7 @@ FactoryBot.define do
admin { true }
end
trait :with_immich_credentials do
trait :with_immich_integration do
settings do
{
immich_url: 'https://immich.example.com',
@ -31,5 +31,14 @@ FactoryBot.define do
}
end
end
trait :with_photoprism_integration do
settings do
{
photoprism_url: 'https://photoprism.example.com',
photoprism_api_key: '1234567890'
}
end
end
end
end

File diff suppressed because one or more lines are too long

View file

@ -6,5 +6,8 @@
2024-03-01T18:33:03Z * {"cog":18,"batt":85,"lon":13.337,"acc":5,"bs":1,"p":100.347,"vel":4,"vac":3,"lat":52.230,"topic":"owntracks/test/iPhone 12 Pro","conn":"m","m":1,"tst":1709317983,"alt":36,"_type":"location","tid":"RO","_http":true}
2024-03-01T18:40:11Z * {"cog":43,"batt":85,"lon":13.338,"acc":5,"bs":1,"p":100.348,"vel":6,"vac":3,"lat":52.231,"topic":"owntracks/test/iPhone 12 Pro","conn":"m","m":1,"tst":1709318411,"alt":37,"_type":"location","tid":"RO","_http":true}
2024-03-01T18:42:57Z * {"cog":320,"batt":85,"lon":13.339,"acc":5,"bs":1,"p":100.353,"vel":3,"vac":3,"lat":52.232,"topic":"owntracks/test/iPhone 12 Pro","t":"v","conn":"m","m":1,"tst":1709318577,"alt":37,"_type":"location","tid":"RO","_http":true}
2024-03-01T18:40:08Z lwt {"_type":"lwt","tst":1717459208}
2024-03-01T18:40:09Z waypoints {"_type":"waypoint","desc":"Home","lat":52.232,"lon":13.339,"rad":50,"tst":1717459768}
2024-03-01T18:40:10Z event {"_type":"transition","acc":5,"desc":"Home","event":"enter","lat":52.232,"lon":13.339,"t":"l","tid":"s8","tst":1717460098,"wtst":1717459768}
2024-03-01T18:40:11Z * {"cog":43,"batt":85,"lon":13.338,"acc":5,"bs":1,"p":100.348,"vel":6,"vac":3,"lat":52.231,"topic":"owntracks/test/iPhone 12 Pro","conn":"m","m":1,"tst":1709318411,"alt":37,"_type":"location","tid":"RO","_http":true}
2024-03-01T18:40:11Z * {"cog":43,"batt":85,"lon":13.341,"acc":5,"bs":1,"p":100.348,"created_at":1709318940,"vel":6,"vac":3,"lat":52.234,"topic":"owntracks/test/iPhone 12 Pro","conn":"m","m":1,"tst":1709318411,"alt":37,"_type":"location","tid":"RO","_http":true}

View file

@ -4,14 +4,17 @@ require 'rails_helper'
RSpec.describe BulkStatsCalculatingJob, type: :job do
describe '#perform' do
it 'enqueues Stats::CalculatingJob for each user' do
user1 = create(:user)
user2 = create(:user)
user3 = create(:user)
let(:user1) { create(:user) }
let(:user2) { create(:user) }
expect(Stats::CalculatingJob).to receive(:perform_later).with(user1.id)
expect(Stats::CalculatingJob).to receive(:perform_later).with(user2.id)
expect(Stats::CalculatingJob).to receive(:perform_later).with(user3.id)
let(:timestamp) { DateTime.new(2024, 1, 1).to_i }
let!(:points1) { create_list(:point, 10, user_id: user1.id, timestamp:) }
let!(:points2) { create_list(:point, 10, user_id: user2.id, timestamp:) }
it 'enqueues Stats::CalculatingJob for each user' do
expect(Stats::CalculatingJob).to receive(:perform_later).with(user1.id, 2024, 1)
expect(Stats::CalculatingJob).to receive(:perform_later).with(user2.id, 2024, 1)
BulkStatsCalculatingJob.perform_now
end

View file

@ -8,16 +8,14 @@ RSpec.describe ImportJob, type: :job do
let(:user) { create(:user) }
let!(:import) { create(:import, user:, name: 'owntracks_export.json') }
let!(:import_points) { create_list(:point, 9, import: import) }
let(:start_at) { Time.zone.at(1_709_283_789) } # Timestamp of the first point in the "2024-03.rec" fixture file
let(:end_at) { import.points.reload.order(:timestamp).last.recorded_at }
it 'creates points' do
expect { perform }.to change { Point.count }.by(9)
end
it 'calls Stats::CalculatingJob' do
expect(Stats::CalculatingJob).to receive(:perform_later).with(user.id, start_at:, end_at:)
# Timestamp of the first point in the "2024-03.rec" fixture file
expect(Stats::CalculatingJob).to receive(:perform_later).with(user.id, 2024, 3)
perform
end

View file

@ -5,24 +5,36 @@ require 'rails_helper'
RSpec.describe Stats::CalculatingJob, type: :job do
describe '#perform' do
let!(:user) { create(:user) }
let(:start_at) { nil }
let(:end_at) { nil }
subject { described_class.perform_now(user.id) }
subject { described_class.perform_now(user.id, 2024, 1) }
before do
allow(Stats::Calculate).to receive(:new).and_call_original
allow_any_instance_of(Stats::Calculate).to receive(:call)
allow(Stats::CalculateMonth).to receive(:new).and_call_original
allow_any_instance_of(Stats::CalculateMonth).to receive(:call)
end
it 'calls Stats::Calculate service' do
it 'calls Stats::CalculateMonth service' do
subject
expect(Stats::Calculate).to have_received(:new).with(user.id, { start_at:, end_at: })
expect(Stats::CalculateMonth).to have_received(:new).with(user.id, 2024, 1)
end
it 'created notifications' do
expect { subject }.to change { Notification.count }.by(1)
context 'when Stats::CalculateMonth raises an error' do
before do
allow_any_instance_of(Stats::CalculateMonth).to receive(:call).and_raise(StandardError)
end
it 'creates an error notification' do
expect { subject }.to change { Notification.count }.by(1)
expect(Notification.last.kind).to eq('error')
end
end
context 'when Stats::CalculateMonth does not raise an error' do
it 'creates an info notification' do
expect { subject }.to change { Notification.count }.by(1)
expect(Notification.last.kind).to eq('info')
end
end
end
end

View file

@ -0,0 +1,37 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe TelemetrySendingJob, type: :job do
describe '#perform' do
let(:gather_service) { instance_double(Telemetry::Gather) }
let(:send_service) { instance_double(Telemetry::Send) }
let(:telemetry_data) { { some: 'data' } }
before do
allow(Telemetry::Gather).to receive(:new).and_return(gather_service)
allow(gather_service).to receive(:call).and_return(telemetry_data)
allow(Telemetry::Send).to receive(:new).with(telemetry_data).and_return(send_service)
allow(send_service).to receive(:call)
end
it 'gathers telemetry data and sends it' do
described_class.perform_now
expect(gather_service).to have_received(:call)
expect(send_service).to have_received(:call)
end
context 'when DISABLE_TELEMETRY is set to true' do
before do
stub_const('ENV', ENV.to_h.merge('DISABLE_TELEMETRY' => 'true'))
end
it 'does not send telemetry data' do
described_class.perform_now
expect(send_service).not_to have_received(:call)
end
end
end
end

View file

@ -17,8 +17,19 @@ RSpec.describe Import, type: :model do
google_phone_takeout: 3,
gpx: 4,
immich_api: 5,
geojson: 6
geojson: 6,
photoprism_api: 7
)
end
end
describe '#years_and_months_tracked' do
let(:import) { create(:import) }
let(:timestamp) { Time.zone.local(2024, 11, 1) }
let!(:points) { create_list(:point, 3, import:, timestamp:) }
it 'returns years and months tracked' do
expect(import.years_and_months_tracked).to eq([[2024, 11]])
end
end
end

View file

@ -51,30 +51,6 @@ RSpec.describe Stat, type: :model do
end
end
describe '.years' do
subject { described_class.years }
context 'when there are no stats' do
it 'returns years' do
expect(subject).to eq([Time.current.year])
end
end
context 'when there are stats' do
let(:user) { create(:user) }
let(:expected_years) { (year..Time.current.year).to_a.reverse }
before do
create(:stat, year: 2021, user:)
create(:stat, year: 2020, user:)
end
it 'returns years' do
expect(subject).to eq(expected_years)
end
end
end
describe '#distance_by_day' do
subject { stat.distance_by_day }
@ -146,5 +122,17 @@ RSpec.describe Stat, type: :model do
end
end
end
describe '#points' do
subject { stat.points.to_a }
let(:stat) { create(:stat, year:, month: 1, user:) }
let(:timestamp) { DateTime.new(year, 1, 1, 5, 0, 0) }
let!(:points) { create_list(:point, 3, user:, timestamp:) }
it 'returns points' do
expect(subject).to eq(points)
end
end
end
end

View file

@ -41,7 +41,7 @@ RSpec.describe Trip, type: :model do
end
end
describe '#photos' do
describe '#photo_previews' do
let(:photo_data) do
[
{
@ -80,8 +80,18 @@ RSpec.describe Trip, type: :model do
let(:trip) { create(:trip, user:) }
let(:expected_photos) do
[
{ url: "/api/v1/photos/456/thumbnail.jpg?api_key=#{user.api_key}" },
{ url: "/api/v1/photos/789/thumbnail.jpg?api_key=#{user.api_key}" }
{
id: '456',
url: "/api/v1/photos/456/thumbnail.jpg?api_key=#{user.api_key}&source=immich",
source: 'immich',
orientation: 'portrait'
},
{
id: '789',
url: "/api/v1/photos/789/thumbnail.jpg?api_key=#{user.api_key}&source=immich",
source: 'immich',
orientation: 'portrait'
}
]
end
@ -93,7 +103,7 @@ RSpec.describe Trip, type: :model do
let(:settings) { {} }
it 'returns an empty array' do
expect(trip.photos).to eq([])
expect(trip.photo_previews).to eq([])
end
end
@ -106,7 +116,9 @@ RSpec.describe Trip, type: :model do
end
it 'returns the photos' do
expect(trip.photos).to eq(expected_photos)
expect(trip.photo_previews).to include(expected_photos[0])
expect(trip.photo_previews).to include(expected_photos[1])
expect(trip.photo_previews.size).to eq(2)
end
end
end

View file

@ -113,5 +113,15 @@ RSpec.describe User, type: :model do
expect(subject).to eq(1)
end
end
describe '#years_tracked' do
let!(:points) { create_list(:point, 3, user:, timestamp: DateTime.new(2024, 1, 1, 5, 0, 0)) }
subject { user.years_tracked }
it 'returns years tracked' do
expect(subject).to eq([2024])
end
end
end
end

View file

@ -4,40 +4,68 @@ require 'rails_helper'
RSpec.describe 'Api::V1::Photos', type: :request do
describe 'GET /index' do
let(:user) { create(:user) }
context 'when the integration is configured' do
let(:user) { create(:user, :with_photoprism_integration) }
let(:photo_data) do
[
{
'id' => '123',
'latitude' => 35.6762,
'longitude' => 139.6503,
'localDateTime' => '2024-01-01T00:00:00.000Z',
'type' => 'photo'
},
{
'id' => '456',
'latitude' => 40.7128,
'longitude' => -74.0060,
'localDateTime' => '2024-01-02T00:00:00.000Z',
'type' => 'photo'
}
]
let(:photo_data) do
[
{
'id' => 1,
'latitude' => 35.6762,
'longitude' => 139.6503,
'localDateTime' => '2024-01-01T00:00:00.000Z',
'originalFileName' => 'photo1.jpg',
'city' => 'Tokyo',
'state' => 'Tokyo',
'country' => 'Japan',
'type' => 'photo',
'source' => 'photoprism'
},
{
'id' => 2,
'latitude' => 40.7128,
'longitude' => -74.0060,
'localDateTime' => '2024-01-02T00:00:00.000Z',
'originalFileName' => 'photo2.jpg',
'city' => 'New York',
'state' => 'New York',
'country' => 'USA',
'type' => 'photo',
'source' => 'immich'
}
]
end
context 'when the request is successful' do
before do
allow_any_instance_of(Photos::Search).to receive(:call).and_return(photo_data)
get '/api/v1/photos', params: { api_key: user.api_key }
end
it 'returns http success' do
expect(response).to have_http_status(:success)
end
it 'returns photos data as JSON' do
expect(JSON.parse(response.body)).to eq(photo_data)
end
end
end
context 'when the request is successful' do
context 'when the integration is not configured' do
let(:user) { create(:user) }
before do
allow_any_instance_of(Immich::RequestPhotos).to receive(:call).and_return(photo_data)
get '/api/v1/photos', params: { api_key: user.api_key }
get '/api/v1/photos', params: { api_key: user.api_key, source: 'immich' }
end
it 'returns http success' do
expect(response).to have_http_status(:success)
it 'returns http unauthorized' do
expect(response).to have_http_status(:unauthorized)
end
it 'returns photos data as JSON' do
expect(JSON.parse(response.body)).to eq(photo_data)
it 'returns an error message' do
expect(JSON.parse(response.body)).to eq({ 'error' => 'Immich integration not configured' })
end
end
end

View file

@ -54,8 +54,14 @@ RSpec.describe '/stats', type: :request do
describe 'POST /update' do
let(:stat) { create(:stat, user:, year: 2024) }
it 'enqueues Stats::CalculatingJob' do
expect { post stats_url(stat.year) }.to have_enqueued_job(Stats::CalculatingJob)
it 'enqueues Stats::CalculatingJob for each tracked year and month' do
allow(user).to receive(:years_tracked).and_return([2024])
post stats_url
(1..12).each do |month|
expect(Stats::CalculatingJob).to have_been_enqueued.with(user.id, 2024, month)
end
end
end
end

View file

@ -25,7 +25,7 @@ RSpec.describe '/trips', type: :request do
stub_request(:any, 'https://api.github.com/repos/Freika/dawarich/tags')
.to_return(status: 200, body: '[{"name": "1.0.0"}]', headers: {})
allow_any_instance_of(Trip).to receive(:photos).and_return([])
allow_any_instance_of(Trip).to receive(:photo_previews).and_return([])
sign_in user
end

View file

@ -0,0 +1,162 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Api::PhotoSerializer do
describe '#call' do
subject(:serialized_photo) { described_class.new(photo, source).call }
context 'when photo is from immich' do
let(:source) { 'immich' }
let(:photo) do
{
"id": '7fe486e3-c3ba-4b54-bbf9-1281b39ed15c',
"deviceAssetId": 'IMG_9913.jpeg-1168914',
"ownerId": 'f579f328-c355-438c-a82c-fe3390bd5f08',
"deviceId": 'CLI',
"libraryId": nil,
"type": 'IMAGE',
"originalPath": 'upload/library/admin/2023/2023-06-08/IMG_9913.jpeg',
"originalFileName": 'IMG_9913.jpeg',
"originalMimeType": 'image/jpeg',
"thumbhash": '4RgONQaZqYaH93g3h3p3d6RfPPrG',
"fileCreatedAt": '2023-06-08T07:58:45.637Z',
"fileModifiedAt": '2023-06-08T09:58:45.000Z',
"localDateTime": '2023-06-08T09:58:45.637Z',
"updatedAt": '2024-08-24T18:20:47.965Z',
"isFavorite": false,
"isArchived": false,
"isTrashed": false,
"duration": '0:00:00.00000',
"exifInfo": {
"make": 'Apple',
"model": 'iPhone 12 Pro',
"exifImageWidth": 4032,
"exifImageHeight": 3024,
"fileSizeInByte": 1_168_914,
"orientation": '6',
"dateTimeOriginal": '2023-06-08T07:58:45.637Z',
"modifyDate": '2023-06-08T07:58:45.000Z',
"timeZone": 'Europe/Berlin',
"lensModel": 'iPhone 12 Pro back triple camera 4.2mm f/1.6',
"fNumber": 1.6,
"focalLength": 4.2,
"iso": 320,
"exposureTime": '1/60',
"latitude": 52.11,
"longitude": 13.22,
"city": 'Johannisthal',
"state": 'Berlin',
"country": 'Germany',
"description": '',
"projectionType": nil,
"rating": nil
},
"livePhotoVideoId": nil,
"people": [],
"checksum": 'aL1edPVg4ZpEnS6xCRWNUY0pUS8=',
"isOffline": false,
"hasMetadata": true,
"duplicateId": '88a34bee-783d-46e4-aa52-33b75ffda375',
"resized": true
}
end
it 'serializes the photo correctly' do
expect(serialized_photo).to eq(
id: '7fe486e3-c3ba-4b54-bbf9-1281b39ed15c',
latitude: 52.11,
longitude: 13.22,
localDateTime: '2023-06-08T09:58:45.637Z',
originalFileName: 'IMG_9913.jpeg',
city: 'Johannisthal',
state: 'Berlin',
country: 'Germany',
type: 'image',
orientation: 'portrait',
source: 'immich'
)
end
end
context 'when photo is from photoprism' do
let(:source) { 'photoprism' }
let(:photo) do
{
'ID' => '102',
'UID' => 'psnver0s3x7wxfnh',
'Type' => 'image',
'TypeSrc' => '',
'TakenAt' => '2023-10-10T16:04:33Z',
'TakenAtLocal' => '2023-10-10T16:04:33Z',
'TakenSrc' => 'name',
'TimeZone' => '',
'Path' => '2023/10',
'Name' => '20231010_160433_91981432',
'OriginalName' => 'photo_2023-10-10 16.04.33',
'Title' => 'Photo / 2023',
'Description' => '',
'Year' => 2023,
'Month' => 10,
'Day' => 10,
'Country' => 'zz',
'Stack' => 0,
'Favorite' => false,
'Private' => false,
'Iso' => 0,
'FocalLength' => 0,
'FNumber' => 0,
'Exposure' => '',
'Quality' => 1,
'Resolution' => 1,
'Color' => 4,
'Scan' => false,
'Panorama' => false,
'CameraID' => 1,
'CameraModel' => 'Unknown',
'LensID' => 1,
'LensModel' => 'Unknown',
'Lat' => 11,
'Lng' => 22,
'CellID' => 'zz',
'PlaceID' => 'zz',
'PlaceSrc' => '',
'PlaceLabel' => 'Unknown',
'PlaceCity' => 'Unknown',
'PlaceState' => 'Unknown',
'PlaceCountry' => 'zz',
'InstanceID' => '',
'FileUID' => 'fsnver0clrfzatmz',
'FileRoot' => '/',
'FileName' => '2023/10/20231010_160433_91981432.jpeg',
'Hash' => 'ce1849fd7cf6a50eb201fbb669ab78c7ac13263b',
'Width' => 1280,
'Height' => 908,
'Portrait' => false,
'Merged' => false,
'CreatedAt' => '2024-12-02T14:25:48Z',
'UpdatedAt' => '2024-12-02T14:36:45Z',
'EditedAt' => '0001-01-01T00:00:00Z',
'CheckedAt' => '2024-12-02T14:36:45Z',
'Files' => nil
}
end
it 'serializes the photo correctly' do
expect(serialized_photo).to eq(
id: 'ce1849fd7cf6a50eb201fbb669ab78c7ac13263b',
latitude: 11,
longitude: 22,
localDateTime: '2023-10-10T16:04:33Z',
originalFileName: 'photo_2023-10-10 16.04.33',
city: 'Unknown',
state: 'Unknown',
country: 'zz',
type: 'image',
orientation: 'landscape',
source: 'photoprism'
)
end
end
end
end

View file

@ -9,7 +9,7 @@ RSpec.describe Immich::RequestPhotos do
let(:user) do
create(:user, settings: { 'immich_url' => 'http://immich.app', 'immich_api_key' => '123456' })
end
let(:immich_data) do
let(:mock_immich_data) do
{
"albums": {
"total": 0,
@ -134,11 +134,11 @@ RSpec.describe Immich::RequestPhotos do
stub_request(
:any,
'http://immich.app/api/search/metadata'
).to_return(status: 200, body: immich_data, headers: {})
).to_return(status: 200, body: mock_immich_data, headers: {})
end
it 'returns images and videos' do
expect(service.map { _1['type'] }.uniq).to eq(['IMAGE', 'VIDEO'])
expect(service.map { _1['type'] }.uniq).to eq(%w[IMAGE VIDEO])
end
end

View file

@ -0,0 +1,113 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Imports::Create do
let(:user) { create(:user) }
let(:service) { described_class.new(user, import) }
describe '#call' do
context 'when source is google_semantic_history' do
let(:import) { create(:import, source: 'google_semantic_history') }
it 'calls the GoogleMaps::SemanticHistoryParser' do
expect(GoogleMaps::SemanticHistoryParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
end
context 'when source is google_phone_takeout' do
let(:import) { create(:import, source: 'google_phone_takeout') }
it 'calls the GoogleMaps::PhoneTakeoutParser' do
expect(GoogleMaps::PhoneTakeoutParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
end
context 'when source is owntracks' do
let(:import) { create(:import, source: 'owntracks') }
it 'calls the OwnTracks::ExportParser' do
expect(OwnTracks::ExportParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
context 'when import is successful' do
it 'creates a finished notification' do
service.call
expect(user.notifications.last.kind).to eq('info')
end
it 'schedules stats creating' do
Sidekiq::Testing.inline! do
expect { service.call }.to \
have_enqueued_job(Stats::CalculatingJob).with(user.id, 2024, 3)
end
end
it 'schedules visit suggesting' do
Sidekiq::Testing.inline! do
expect { service.call }.to have_enqueued_job(VisitSuggestingJob)
end
end
end
context 'when import fails' do
before do
allow(OwnTracks::ExportParser).to receive(:new).with(import, user.id).and_return(double(call: false))
end
it 'creates a failed notification' do
service.call
expect(user.notifications.last.kind).to eq('error')
end
end
end
context 'when source is gpx' do
let(:import) { create(:import, source: 'gpx') }
it 'calls the Gpx::TrackParser' do
expect(Gpx::TrackParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
end
context 'when source is geojson' do
let(:import) { create(:import, source: 'geojson') }
it 'calls the Geojson::ImportParser' do
expect(Geojson::ImportParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
end
context 'when source is immich_api' do
let(:import) { create(:import, source: 'immich_api') }
it 'calls the Photos::ImportParser' do
expect(Photos::ImportParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
end
context 'when source is photoprism_api' do
let(:import) { create(:import, source: 'photoprism_api') }
it 'calls the Photos::ImportParser' do
expect(Photos::ImportParser).to \
receive(:new).with(import, user.id).and_return(double(call: true))
service.call
end
end
end
end

View file

@ -0,0 +1,19 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Photoprism::CachePreviewToken, type: :service do
let(:user) { double('User', id: 1) }
let(:preview_token) { 'sample_token' }
let(:service) { described_class.new(user, preview_token) }
describe '#call' do
it 'writes the preview token to the cache with the correct key' do
expect(Rails.cache).to receive(:write).with(
"dawarich/photoprism_preview_token_#{user.id}", preview_token
)
service.call
end
end
end

View file

@ -0,0 +1,177 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Photoprism::ImportGeodata do
describe '#call' do
subject(:service) { described_class.new(user).call }
let(:user) do
create(:user, settings: { 'photoprism_url' => 'http://photoprism.app', 'photoprism_api_key' => '123456' })
end
let(:photoprism_data) do
[
{
'ID' => '82',
'UID' => 'psnveqq089xhy1c3',
'Type' => 'image',
'TypeSrc' => '',
'TakenAt' => '2024-08-18T14:11:05Z',
'TakenAtLocal' => '2024-08-18T16:11:05Z',
'TakenSrc' => 'meta',
'TimeZone' => 'Europe/Prague',
'Path' => '2024/08',
'Name' => '20240818_141105_44E61AED',
'OriginalName' => 'PXL_20240818_141105789',
'Title' => 'Moment / Karlovy Vary / 2024',
'Description' => '',
'Year' => 2024,
'Month' => 8,
'Day' => 18,
'Country' => 'cz',
'Stack' => 0,
'Favorite' => false,
'Private' => false,
'Iso' => 37,
'FocalLength' => 21,
'FNumber' => 2.2,
'Exposure' => '1/347',
'Quality' => 4,
'Resolution' => 10,
'Color' => 2,
'Scan' => false,
'Panorama' => false,
'CameraID' => 8,
'CameraSrc' => 'meta',
'CameraMake' => 'Google',
'CameraModel' => 'Pixel 7 Pro',
'LensID' => 11,
'LensMake' => 'Google',
'LensModel' => 'Pixel 7 Pro front camera 2.74mm f/2.2',
'Altitude' => 423,
'Lat' => 50.11,
'Lng' => 12.12,
'CellID' => 's2:47a09944f33c',
'PlaceID' => 'cz:ciNqTjWuq6NN',
'PlaceSrc' => 'meta',
'PlaceLabel' => 'Karlovy Vary, Severozápad, Czech Republic',
'PlaceCity' => 'Karlovy Vary',
'PlaceState' => 'Severozápad',
'PlaceCountry' => 'cz',
'InstanceID' => '',
'FileUID' => 'fsnveqqeusn692qo',
'FileRoot' => '/',
'FileName' => '2024/08/20240818_141105_44E61AED.jpg',
'Hash' => 'cc5d0f544e52b288d7c8460d2e1bb17fa66e6089',
'Width' => 2736,
'Height' => 3648,
'Portrait' => true,
'Merged' => false,
'CreatedAt' => '2024-12-02T14:25:38Z',
'UpdatedAt' => '2024-12-02T14:25:38Z',
'EditedAt' => '0001-01-01T00:00:00Z',
'CheckedAt' => '2024-12-02T14:36:45Z',
'Files' => nil
},
{
'ID' => '81',
'UID' => 'psnveqpl96gcfdzf',
'Type' => 'image',
'TypeSrc' => '',
'TakenAt' => '2024-08-18T14:11:04Z',
'TakenAtLocal' => '2024-08-18T16:11:04Z',
'TakenSrc' => 'meta',
'TimeZone' => 'Europe/Prague',
'Path' => '2024/08',
'Name' => '20240818_141104_E9949CD4',
'OriginalName' => 'PXL_20240818_141104633',
'Title' => 'Portrait / Karlovy Vary / 2024',
'Description' => '',
'Year' => 2024,
'Month' => 8,
'Day' => 18,
'Country' => 'cz',
'Stack' => 0,
'Favorite' => false,
'Private' => false,
'Iso' => 43,
'FocalLength' => 21,
'FNumber' => 2.2,
'Exposure' => '1/356',
'Faces' => 1,
'Quality' => 4,
'Resolution' => 10,
'Color' => 2,
'Scan' => false,
'Panorama' => false,
'CameraID' => 8,
'CameraSrc' => 'meta',
'CameraMake' => 'Google',
'CameraModel' => 'Pixel 7 Pro',
'LensID' => 11,
'LensMake' => 'Google',
'LensModel' => 'Pixel 7 Pro front camera 2.74mm f/2.2',
'Altitude' => 423,
'Lat' => 50.21,
'Lng' => 12.85,
'CellID' => 's2:47a09944f33c',
'PlaceID' => 'cz:ciNqTjWuq6NN',
'PlaceSrc' => 'meta',
'PlaceLabel' => 'Karlovy Vary, Severozápad, Czech Republic',
'PlaceCity' => 'Karlovy Vary',
'PlaceState' => 'Severozápad',
'PlaceCountry' => 'cz',
'InstanceID' => '',
'FileUID' => 'fsnveqp9xsl7onsv',
'FileRoot' => '/',
'FileName' => '2024/08/20240818_141104_E9949CD4.jpg',
'Hash' => 'd5dfadc56a0b63051dfe0b5dec55ff1d81f033b7',
'Width' => 2736,
'Height' => 3648,
'Portrait' => true,
'Merged' => false,
'CreatedAt' => '2024-12-02T14:25:37Z',
'UpdatedAt' => '2024-12-02T14:25:37Z',
'EditedAt' => '0001-01-01T00:00:00Z',
'CheckedAt' => '2024-12-02T14:36:45Z',
'Files' => nil
}
].to_json
end
before do
stub_request(:get, %r{http://photoprism\.app/api/v1/photos}).with(
headers: {
'Accept' => 'application/json',
'Accept-Encoding' => 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
'Authorization' => 'Bearer 123456',
'User-Agent' => 'Ruby'
}
).to_return(status: 200, body: photoprism_data, headers: {})
end
it 'creates import' do
expect { service }.to change { Import.count }.by(1)
end
it 'enqueues ImportJob' do
expect(ImportJob).to receive(:perform_later)
service
end
context 'when import already exists' do
before { service }
it 'does not create new import' do
expect { service }.not_to(change { Import.count })
end
it 'does not enqueue ImportJob' do
expect(ImportJob).to_not receive(:perform_later)
service
end
end
end
end

View file

@ -0,0 +1,287 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Photoprism::RequestPhotos do
let(:user) do
create(
:user,
settings: {
'photoprism_url' => 'http://photoprism.local',
'photoprism_api_key' => 'test_api_key'
}
)
end
let(:start_date) { '2024-01-01' }
let(:end_date) { '2024-12-31' }
let(:service) { described_class.new(user, start_date: start_date, end_date: end_date) }
let(:mock_photo_response) do
[
{
'ID' => '82',
'UID' => 'psnveqq089xhy1c3',
'Type' => 'image',
'TypeSrc' => '',
'TakenAt' => '2024-08-18T14:11:05Z',
'TakenAtLocal' => '2024-08-18T16:11:05Z',
'TakenSrc' => 'meta',
'TimeZone' => 'Europe/Prague',
'Path' => '2024/08',
'Name' => '20240818_141105_44E61AED',
'OriginalName' => 'PXL_20240818_141105789',
'Title' => 'Moment / Karlovy Vary / 2024',
'Description' => '',
'Year' => 2024,
'Month' => 8,
'Day' => 18,
'Country' => 'cz',
'Stack' => 0,
'Favorite' => false,
'Private' => false,
'Iso' => 37,
'FocalLength' => 21,
'FNumber' => 2.2,
'Exposure' => '1/347',
'Quality' => 4,
'Resolution' => 10,
'Color' => 2,
'Scan' => false,
'Panorama' => false,
'CameraID' => 8,
'CameraSrc' => 'meta',
'CameraMake' => 'Google',
'CameraModel' => 'Pixel 7 Pro',
'LensID' => 11,
'LensMake' => 'Google',
'LensModel' => 'Pixel 7 Pro front camera 2.74mm f/2.2',
'Altitude' => 423,
'Lat' => 50.11,
'Lng' => 12.12,
'CellID' => 's2:47a09944f33c',
'PlaceID' => 'cz:ciNqTjWuq6NN',
'PlaceSrc' => 'meta',
'PlaceLabel' => 'Karlovy Vary, Severozápad, Czech Republic',
'PlaceCity' => 'Karlovy Vary',
'PlaceState' => 'Severozápad',
'PlaceCountry' => 'cz',
'InstanceID' => '',
'FileUID' => 'fsnveqqeusn692qo',
'FileRoot' => '/',
'FileName' => '2024/08/20240818_141105_44E61AED.jpg',
'Hash' => 'cc5d0f544e52b288d7c8460d2e1bb17fa66e6089',
'Width' => 2736,
'Height' => 3648,
'Portrait' => true,
'Merged' => false,
'CreatedAt' => '2024-12-02T14:25:38Z',
'UpdatedAt' => '2024-12-02T14:25:38Z',
'EditedAt' => '0001-01-01T00:00:00Z',
'CheckedAt' => '2024-12-02T14:36:45Z',
'Files' => nil
},
{
'ID' => '81',
'UID' => 'psnveqpl96gcfdzf',
'Type' => 'image',
'TypeSrc' => '',
'TakenAt' => '2024-08-18T14:11:04Z',
'TakenAtLocal' => '2024-08-18T16:11:04Z',
'TakenSrc' => 'meta',
'TimeZone' => 'Europe/Prague',
'Path' => '2024/08',
'Name' => '20240818_141104_E9949CD4',
'OriginalName' => 'PXL_20240818_141104633',
'Title' => 'Portrait / Karlovy Vary / 2024',
'Description' => '',
'Year' => 2024,
'Month' => 8,
'Day' => 18,
'Country' => 'cz',
'Stack' => 0,
'Favorite' => false,
'Private' => false,
'Iso' => 43,
'FocalLength' => 21,
'FNumber' => 2.2,
'Exposure' => '1/356',
'Faces' => 1,
'Quality' => 4,
'Resolution' => 10,
'Color' => 2,
'Scan' => false,
'Panorama' => false,
'CameraID' => 8,
'CameraSrc' => 'meta',
'CameraMake' => 'Google',
'CameraModel' => 'Pixel 7 Pro',
'LensID' => 11,
'LensMake' => 'Google',
'LensModel' => 'Pixel 7 Pro front camera 2.74mm f/2.2',
'Altitude' => 423,
'Lat' => 50.21,
'Lng' => 12.85,
'CellID' => 's2:47a09944f33c',
'PlaceID' => 'cz:ciNqTjWuq6NN',
'PlaceSrc' => 'meta',
'PlaceLabel' => 'Karlovy Vary, Severozápad, Czech Republic',
'PlaceCity' => 'Karlovy Vary',
'PlaceState' => 'Severozápad',
'PlaceCountry' => 'cz',
'InstanceID' => '',
'FileUID' => 'fsnveqp9xsl7onsv',
'FileRoot' => '/',
'FileName' => '2024/08/20240818_141104_E9949CD4.jpg',
'Hash' => 'd5dfadc56a0b63051dfe0b5dec55ff1d81f033b7',
'Width' => 2736,
'Height' => 3648,
'Portrait' => true,
'Merged' => false,
'CreatedAt' => '2024-12-02T14:25:37Z',
'UpdatedAt' => '2024-12-02T14:25:37Z',
'EditedAt' => '0001-01-01T00:00:00Z',
'CheckedAt' => '2024-12-02T14:36:45Z',
'Files' => nil
}
]
end
describe '#call' do
context 'with valid credentials' do
before do
stub_request(
:any,
"#{user.settings['photoprism_url']}/api/v1/photos?after=#{start_date}&before=#{end_date}&count=1000&public=true&q=&quality=3"
).with(
headers: {
'Accept' => 'application/json',
'Accept-Encoding' => 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
'Authorization' => 'Bearer test_api_key',
'User-Agent' => 'Ruby'
}
).to_return(
status: 200,
body: mock_photo_response.to_json,
headers: { 'Content-Type' => 'application/json' }
)
stub_request(
:any,
"#{user.settings['photoprism_url']}/api/v1/photos?after=#{start_date}&before=#{end_date}&count=1000&public=true&q=&quality=3&offset=1000"
).to_return(status: 200, body: [].to_json)
end
it 'returns photos within the specified date range' do
result = service.call
expect(result).to be_an(Array)
expect(result.first['Title']).to eq('Moment / Karlovy Vary / 2024')
end
end
context 'with missing credentials' do
let(:user) { create(:user, settings: {}) }
it 'raises error when Photoprism URL is missing' do
expect { service.call }.to raise_error(ArgumentError, 'Photoprism URL is missing')
end
it 'raises error when API key is missing' do
user.update(settings: { 'photoprism_url' => 'http://photoprism.local' })
expect { service.call }.to raise_error(ArgumentError, 'Photoprism API key is missing')
end
end
context 'when API returns an error' do
before do
stub_request(
:get,
"#{user.settings['photoprism_url']}/api/v1/photos?after=#{start_date}&before=#{end_date}&count=1000&public=true&q=&quality=3"
).to_return(status: 400, body: { status: 400, error: 'Unable to do that' }.to_json)
end
it 'logs the error' do
expect(Rails.logger).to \
receive(:error).with('Photoprism API returned 400: {"status":400,"error":"Unable to do that"}')
expect(Rails.logger).to \
receive(:debug).with(
"Photoprism API request params: #{{ q: '', public: true, quality: 3, after: start_date, count: 1000,
before: end_date }}"
)
service.call
end
end
context 'with pagination' do
let(:first_page) { [{ 'TakenAtLocal' => "#{start_date}T14:30:00Z" }] }
let(:second_page) { [{ 'TakenAtLocal' => "#{start_date}T14:30:00Z" }] }
let(:empty_page) { [] }
let(:common_headers) do
{
'Accept' => 'application/json',
'Accept-Encoding' => 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
'Authorization' => 'Bearer test_api_key',
'User-Agent' => 'Ruby'
}
end
before do
# First page
stub_request(:any, "#{user.settings['photoprism_url']}/api/v1/photos")
.with(
headers: common_headers,
query: {
after: start_date,
before: end_date,
count: '1000',
public: 'true',
q: '',
quality: '3'
}
)
.to_return(status: 200, body: first_page.to_json)
# Second page
stub_request(:any, "#{user.settings['photoprism_url']}/api/v1/photos")
.with(
headers: common_headers,
query: {
after: start_date,
before: end_date,
count: '1000',
public: 'true',
q: '',
quality: '3',
offset: '1000'
}
)
.to_return(status: 200, body: second_page.to_json)
# Last page (empty)
stub_request(:any, "#{user.settings['photoprism_url']}/api/v1/photos")
.with(
headers: common_headers,
query: {
after: start_date,
before: end_date,
count: '1000',
public: 'true',
q: '',
quality: '3',
offset: '2000'
}
)
.to_return(status: 200, body: empty_page.to_json)
end
it 'fetches all pages until empty result' do
result = service.call
expect(result.size).to eq(2)
end
end
end
end

View file

@ -2,7 +2,7 @@
require 'rails_helper'
RSpec.describe Immich::ImportParser do
RSpec.describe Photos::ImportParser do
describe '#call' do
subject(:service) { described_class.new(import, user.id).call }

View file

@ -0,0 +1,149 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Photos::Search do
let(:user) { create(:user) }
let(:start_date) { '2024-01-01' }
let(:end_date) { '2024-03-01' }
let(:service) { described_class.new(user, start_date: start_date, end_date: end_date) }
describe '#call' do
context 'when user has no integrations configured' do
before do
allow(user).to receive(:immich_integration_configured?).and_return(false)
allow(user).to receive(:photoprism_integration_configured?).and_return(false)
end
it 'returns an empty array' do
expect(service.call).to eq([])
end
end
context 'when user has Immich integration configured' do
let(:immich_photo) { { 'type' => 'image', 'id' => '1' } }
let(:serialized_photo) { { id: '1', source: 'immich' } }
before do
allow(user).to receive(:immich_integration_configured?).and_return(true)
allow(user).to receive(:photoprism_integration_configured?).and_return(false)
allow_any_instance_of(Immich::RequestPhotos).to receive(:call)
.and_return([immich_photo])
allow_any_instance_of(Api::PhotoSerializer).to receive(:call)
.and_return(serialized_photo)
end
it 'fetches and transforms Immich photos' do
expect(service.call).to eq([serialized_photo])
end
end
context 'when user has Photoprism integration configured' do
let(:photoprism_photo) { { 'Type' => 'image', 'id' => '2' } }
let(:serialized_photo) { { id: '2', source: 'photoprism' } }
before do
allow(user).to receive(:immich_integration_configured?).and_return(false)
allow(user).to receive(:photoprism_integration_configured?).and_return(true)
allow_any_instance_of(Photoprism::RequestPhotos).to receive(:call)
.and_return([photoprism_photo])
allow_any_instance_of(Api::PhotoSerializer).to receive(:call)
.and_return(serialized_photo)
end
it 'fetches and transforms Photoprism photos' do
expect(service.call).to eq([serialized_photo])
end
end
context 'when user has both integrations configured' do
let(:immich_photo) { { 'type' => 'image', 'id' => '1' } }
let(:photoprism_photo) { { 'Type' => 'image', 'id' => '2' } }
let(:serialized_immich) do
{
id: '1',
latitude: nil,
longitude: nil,
localDateTime: nil,
originalFileName: nil,
city: nil,
state: nil,
country: nil,
type: 'image',
source: 'immich',
orientation: 'landscape'
}
end
let(:serialized_photoprism) do
{
id: '2',
latitude: nil,
longitude: nil,
localDateTime: nil,
originalFileName: nil,
city: nil,
state: nil,
country: nil,
type: 'image',
source: 'photoprism',
orientation: 'landscape'
}
end
before do
allow(user).to receive(:immich_integration_configured?).and_return(true)
allow(user).to receive(:photoprism_integration_configured?).and_return(true)
allow_any_instance_of(Immich::RequestPhotos).to receive(:call)
.and_return([immich_photo])
allow_any_instance_of(Photoprism::RequestPhotos).to receive(:call)
.and_return([photoprism_photo])
end
it 'fetches and transforms photos from both services' do
expect(service.call).to eq([serialized_immich, serialized_photoprism])
end
end
context 'when filtering out videos' do
let(:immich_photo) { { 'type' => 'video', 'id' => '1' } }
before do
allow(user).to receive(:immich_integration_configured?).and_return(true)
allow(user).to receive(:photoprism_integration_configured?).and_return(false)
allow_any_instance_of(Immich::RequestPhotos).to receive(:call)
.and_return([immich_photo])
end
it 'excludes video assets' do
expect(service.call).to eq([])
end
end
end
describe '#initialize' do
context 'with default parameters' do
let(:service_default) { described_class.new(user) }
it 'sets default start_date' do
expect(service_default.start_date).to eq('1970-01-01')
end
it 'sets default end_date to nil' do
expect(service_default.end_date).to be_nil
end
end
context 'with custom parameters' do
it 'sets custom dates' do
expect(service.start_date).to eq(start_date)
expect(service.end_date).to eq(end_date)
end
end
end
end

View file

@ -0,0 +1,77 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Photos::Thumbnail do
let(:user) { create(:user) }
let(:id) { 'photo123' }
describe '#call' do
subject { described_class.new(user, source, id).call }
context 'with immich source' do
let(:source) { 'immich' }
let(:api_key) { 'immich_key_123' }
let(:base_url) { 'https://photos.example.com' }
let(:expected_url) { "#{base_url}/api/assets/#{id}/thumbnail?size=preview" }
let(:expected_headers) do
{
'accept' => 'application/octet-stream',
'X-Api-Key' => api_key
}
end
before do
allow(user).to receive(:settings).and_return(
'immich_url' => base_url,
'immich_api_key' => api_key
)
end
it 'fetches thumbnail with correct parameters' do
expect(HTTParty).to receive(:get)
.with(expected_url, headers: expected_headers)
.and_return('thumbnail_data')
expect(subject).to eq('thumbnail_data')
end
end
context 'with photoprism source' do
let(:source) { 'photoprism' }
let(:base_url) { 'https://photoprism.example.com' }
let(:preview_token) { 'preview_token_123' }
let(:expected_url) { "#{base_url}/api/v1/t/#{id}/#{preview_token}/tile_500" }
let(:expected_headers) do
{
'accept' => 'application/octet-stream'
}
end
before do
allow(user).to receive(:settings).and_return(
'photoprism_url' => base_url
)
allow(Rails.cache).to receive(:read)
.with("#{Photoprism::CachePreviewToken::TOKEN_CACHE_KEY}_#{user.id}")
.and_return(preview_token)
end
it 'fetches thumbnail with correct parameters' do
expect(HTTParty).to receive(:get)
.with(expected_url, headers: expected_headers)
.and_return('thumbnail_data')
expect(subject).to eq('thumbnail_data')
end
end
context 'with unsupported source' do
let(:source) { 'unsupported' }
it 'raises an error' do
expect { subject }.to raise_error(RuntimeError, 'Unsupported source: unsupported')
end
end
end
end

View file

@ -2,11 +2,13 @@
require 'rails_helper'
RSpec.describe Stats::Calculate do
RSpec.describe Stats::CalculateMonth do
describe '#call' do
subject(:calculate_stats) { described_class.new(user.id).call }
subject(:calculate_stats) { described_class.new(user.id, year, month).call }
let(:user) { create(:user) }
let(:year) { 2021 }
let(:month) { 1 }
context 'when there are no points' do
it 'does not create stats' do
@ -15,9 +17,9 @@ RSpec.describe Stats::Calculate do
end
context 'when there are points' do
let(:timestamp1) { DateTime.new(2021, 1, 1, 12).to_i }
let(:timestamp2) { DateTime.new(2021, 1, 1, 13).to_i }
let(:timestamp3) { DateTime.new(2021, 1, 1, 14).to_i }
let(:timestamp1) { DateTime.new(year, month, 1, 12).to_i }
let(:timestamp2) { DateTime.new(year, month, 1, 13).to_i }
let(:timestamp3) { DateTime.new(year, month, 1, 14).to_i }
let!(:import) { create(:import, user:) }
let!(:point1) do
create(:point,

View file

@ -0,0 +1,45 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Telemetry::Gather do
let!(:user) { create(:user, last_sign_in_at: Time.zone.today) }
describe '#call' do
subject(:gather) { described_class.new.call }
it 'returns a hash with measurement, timestamp, tags, and fields' do
expect(gather).to include(:measurement, :timestamp, :tags, :fields)
end
it 'includes the correct measurement' do
expect(gather[:measurement]).to eq('dawarich_usage_metrics')
end
it 'includes the current timestamp' do
expect(gather[:timestamp]).to be_within(1).of(Time.current.to_i)
end
it 'includes the correct instance_id in tags' do
expect(gather[:tags][:instance_id]).to eq(Digest::SHA2.hexdigest(user.api_key))
end
it 'includes the correct app_version in fields' do
expect(gather[:fields][:app_version]).to eq("\"#{APP_VERSION}\"")
end
it 'includes the correct dau in fields' do
expect(gather[:fields][:dau]).to eq(1)
end
context 'with a custom measurement' do
let(:measurement) { 'custom_measurement' }
subject(:gather) { described_class.new(measurement:).call }
it 'includes the correct measurement' do
expect(gather[:measurement]).to eq('custom_measurement')
end
end
end
end

View file

@ -0,0 +1,72 @@
# frozen_string_literal: true
require 'rails_helper'
RSpec.describe Trips::Photos do
let(:user) { instance_double('User') }
let(:trip) { instance_double('Trip', started_at: Date.new(2024, 1, 1), ended_at: Date.new(2024, 1, 7)) }
let(:service) { described_class.new(trip, user) }
describe '#call' do
context 'when user has no photo integrations configured' do
before do
allow(user).to receive(:immich_integration_configured?).and_return(false)
allow(user).to receive(:photoprism_integration_configured?).and_return(false)
end
it 'returns an empty array' do
expect(service.call).to eq([])
end
end
context 'when user has photo integrations configured' do
let(:photo_search) { instance_double('Photos::Search') }
let(:raw_photos) do
[
{
id: 1,
url: '/api/v1/photos/1/thumbnail.jpg?api_key=test-api-key&source=immich',
source: 'immich',
orientation: 'landscape'
},
{
id: 2,
url: '/api/v1/photos/2/thumbnail.jpg?api_key=test-api-key&source=photoprism',
source: 'photoprism',
orientation: 'portrait'
}
]
end
before do
allow(user).to receive(:immich_integration_configured?).and_return(true)
allow(user).to receive(:photoprism_integration_configured?).and_return(false)
allow(user).to receive(:api_key).and_return('test-api-key')
allow(Photos::Search).to receive(:new)
.with(user, start_date: '2024-01-01', end_date: '2024-01-07')
.and_return(photo_search)
allow(photo_search).to receive(:call).and_return(raw_photos)
end
it 'returns formatted photo thumbnails' do
expected_result = [
{
id: 1,
url: '/api/v1/photos/1/thumbnail.jpg?api_key=test-api-key&source=immich',
source: 'immich',
orientation: 'landscape'
},
{
id: 2,
url: '/api/v1/photos/2/thumbnail.jpg?api_key=test-api-key&source=photoprism',
source: 'photoprism',
orientation: 'portrait'
}
]
expect(service.call).to eq(expected_result)
end
end
end
end

View file

@ -3,7 +3,7 @@
require 'swagger_helper'
RSpec.describe 'Api::V1::PhotosController', type: :request do
let(:user) { create(:user, :with_immich_credentials) }
let(:user) { create(:user, :with_immich_integration) }
let(:api_key) { user.api_key }
let(:start_date) { '2024-01-01' }
let(:end_date) { '2024-01-02' }
@ -103,59 +103,18 @@ RSpec.describe 'Api::V1::PhotosController', type: :request do
type: :object,
properties: {
id: { type: :string },
deviceAssetId: { type: :string },
ownerId: { type: :string },
type: { type: :string },
originalPath: { type: :string },
originalFileName: { type: :string },
originalMimeType: { type: :string },
thumbhash: { type: :string },
fileCreatedAt: { type: :string, format: 'date-time' },
fileModifiedAt: { type: :string, format: 'date-time' },
latitude: { type: :number, format: :float },
longitude: { type: :number, format: :float },
localDateTime: { type: :string, format: 'date-time' },
updatedAt: { type: :string, format: 'date-time' },
isFavorite: { type: :boolean },
isArchived: { type: :boolean },
isTrashed: { type: :boolean },
duration: { type: :string },
exifInfo: {
type: :object,
properties: {
make: { type: :string },
model: { type: :string },
exifImageWidth: { type: :integer },
exifImageHeight: { type: :integer },
fileSizeInByte: { type: :integer },
orientation: { type: :string },
dateTimeOriginal: { type: :string, format: 'date-time' },
modifyDate: { type: :string, format: 'date-time' },
timeZone: { type: :string },
lensModel: { type: :string },
fNumber: { type: :number, format: :float },
focalLength: { type: :number, format: :float },
iso: { type: :integer },
exposureTime: { type: :string },
latitude: { type: :number, format: :float },
longitude: { type: :number, format: :float },
city: { type: :string },
state: { type: :string },
country: { type: :string },
description: { type: :string },
projectionType: { type: %i[string null] },
rating: { type: %i[integer null] }
}
},
checksum: { type: :string },
isOffline: { type: :boolean },
hasMetadata: { type: :boolean },
duplicateId: { type: :string },
resized: { type: :boolean }
originalFileName: { type: :string },
city: { type: :string },
state: { type: :string },
country: { type: :string },
type: { type: :string, enum: %w[image video] },
orientation: { type: :string, enum: %w[portrait landscape] },
source: { type: :string, enum: %w[immich photoprism] }
},
required: %w[id deviceAssetId ownerId type originalPath
originalFileName originalMimeType thumbhash
fileCreatedAt fileModifiedAt localDateTime
updatedAt isFavorite isArchived isTrashed duration
exifInfo checksum isOffline hasMetadata duplicateId resized]
required: %w[id latitude longitude localDateTime originalFileName city state country type source]
}
run_test! do |response|
@ -172,61 +131,25 @@ RSpec.describe 'Api::V1::PhotosController', type: :request do
produces 'application/json'
parameter name: :id, in: :path, type: :string, required: true
parameter name: :api_key, in: :query, type: :string, required: true
parameter name: :source, in: :query, type: :string, required: true
response '200', 'photo found' do
schema type: :object,
properties: {
id: { type: :string },
deviceAssetId: { type: :string },
ownerId: { type: :string },
type: { type: :string },
originalPath: { type: :string },
originalFileName: { type: :string },
originalMimeType: { type: :string },
thumbhash: { type: :string },
fileCreatedAt: { type: :string, format: 'date-time' },
fileModifiedAt: { type: :string, format: 'date-time' },
latitude: { type: :number, format: :float },
longitude: { type: :number, format: :float },
localDateTime: { type: :string, format: 'date-time' },
updatedAt: { type: :string, format: 'date-time' },
isFavorite: { type: :boolean },
isArchived: { type: :boolean },
isTrashed: { type: :boolean },
duration: { type: :string },
exifInfo: {
type: :object,
properties: {
make: { type: :string },
model: { type: :string },
exifImageWidth: { type: :integer },
exifImageHeight: { type: :integer },
fileSizeInByte: { type: :integer },
orientation: { type: :string },
dateTimeOriginal: { type: :string, format: 'date-time' },
modifyDate: { type: :string, format: 'date-time' },
timeZone: { type: :string },
lensModel: { type: :string },
fNumber: { type: :number, format: :float },
focalLength: { type: :number, format: :float },
iso: { type: :integer },
exposureTime: { type: :string },
latitude: { type: :number, format: :float },
longitude: { type: :number, format: :float },
city: { type: :string },
state: { type: :string },
country: { type: :string },
description: { type: :string },
projectionType: { type: %i[string null] },
rating: { type: %i[integer null] }
}
},
checksum: { type: :string },
isOffline: { type: :boolean },
hasMetadata: { type: :boolean },
duplicateId: { type: :string },
resized: { type: :boolean }
originalFileName: { type: :string },
city: { type: :string },
state: { type: :string },
country: { type: :string },
type: { type: :string, enum: %w[IMAGE VIDEO image video raw live animated] },
orientation: { type: :string, enum: %w[portrait landscape] },
source: { type: :string, enum: %w[immich photoprism] }
}
let(:id) { '7fe486e3-c3ba-4b54-bbf9-1281b39ed15c' }
let(:source) { 'immich' }
run_test! do |response|
data = JSON.parse(response.body)
@ -238,6 +161,7 @@ RSpec.describe 'Api::V1::PhotosController', type: :request do
response '404', 'photo not found' do
let(:id) { 'nonexistent' }
let(:api_key) { user.api_key }
let(:source) { 'immich' }
run_test! do |response|
data = JSON.parse(response.body)

View file

@ -347,130 +347,49 @@ paths:
properties:
id:
type: string
deviceAssetId:
type: string
ownerId:
type: string
type:
type: string
originalPath:
type: string
originalFileName:
type: string
originalMimeType:
type: string
thumbhash:
type: string
fileCreatedAt:
type: string
format: date-time
fileModifiedAt:
type: string
format: date-time
latitude:
type: number
format: float
longitude:
type: number
format: float
localDateTime:
type: string
format: date-time
updatedAt:
originalFileName:
type: string
format: date-time
isFavorite:
type: boolean
isArchived:
type: boolean
isTrashed:
type: boolean
duration:
city:
type: string
exifInfo:
type: object
properties:
make:
type: string
model:
type: string
exifImageWidth:
type: integer
exifImageHeight:
type: integer
fileSizeInByte:
type: integer
orientation:
type: string
dateTimeOriginal:
type: string
format: date-time
modifyDate:
type: string
format: date-time
timeZone:
type: string
lensModel:
type: string
fNumber:
type: number
format: float
focalLength:
type: number
format: float
iso:
type: integer
exposureTime:
type: string
latitude:
type: number
format: float
longitude:
type: number
format: float
city:
type: string
state:
type: string
country:
type: string
description:
type: string
projectionType:
type:
- string
- 'null'
rating:
type:
- integer
- 'null'
checksum:
state:
type: string
isOffline:
type: boolean
hasMetadata:
type: boolean
duplicateId:
country:
type: string
resized:
type: boolean
type:
type: string
enum:
- image
- video
orientation:
type: string
enum:
- portrait
- landscape
source:
type: string
enum:
- immich
- photoprism
required:
- id
- deviceAssetId
- ownerId
- type
- originalPath
- originalFileName
- originalMimeType
- thumbhash
- fileCreatedAt
- fileModifiedAt
- latitude
- longitude
- localDateTime
- updatedAt
- isFavorite
- isArchived
- isTrashed
- duration
- exifInfo
- checksum
- isOffline
- hasMetadata
- duplicateId
- resized
- originalFileName
- city
- state
- country
- type
- source
"/api/v1/photos/{id}/thumbnail":
get:
summary: Retrieves a photo
@ -487,6 +406,11 @@ paths:
required: true
schema:
type: string
- name: source
in: query
required: true
schema:
type: string
responses:
'200':
description: photo found
@ -497,107 +421,43 @@ paths:
properties:
id:
type: string
deviceAssetId:
type: string
ownerId:
type: string
type:
type: string
originalPath:
type: string
originalFileName:
type: string
originalMimeType:
type: string
thumbhash:
type: string
fileCreatedAt:
type: string
format: date-time
fileModifiedAt:
type: string
format: date-time
latitude:
type: number
format: float
longitude:
type: number
format: float
localDateTime:
type: string
format: date-time
updatedAt:
originalFileName:
type: string
format: date-time
isFavorite:
type: boolean
isArchived:
type: boolean
isTrashed:
type: boolean
duration:
city:
type: string
exifInfo:
type: object
properties:
make:
type: string
model:
type: string
exifImageWidth:
type: integer
exifImageHeight:
type: integer
fileSizeInByte:
type: integer
orientation:
type: string
dateTimeOriginal:
type: string
format: date-time
modifyDate:
type: string
format: date-time
timeZone:
type: string
lensModel:
type: string
fNumber:
type: number
format: float
focalLength:
type: number
format: float
iso:
type: integer
exposureTime:
type: string
latitude:
type: number
format: float
longitude:
type: number
format: float
city:
type: string
state:
type: string
country:
type: string
description:
type: string
projectionType:
type:
- string
- 'null'
rating:
type:
- integer
- 'null'
checksum:
state:
type: string
isOffline:
type: boolean
hasMetadata:
type: boolean
duplicateId:
country:
type: string
resized:
type: boolean
type:
type: string
enum:
- IMAGE
- VIDEO
- image
- video
- raw
- live
- animated
orientation:
type: string
enum:
- portrait
- landscape
source:
type: string
enum:
- immich
- photoprism
'404':
description: photo not found
"/api/v1/points":