[BUG] App re-uploads everything after reinstall

[dorianim]

Describe the bug
I just had to reinstall the immich app, because I switched from the f-droid version to the apk from the GitHub release.
When I logged in again, all photos and videos are being re-uploaded. The App thinks, they are all new.
The number of Total assets also changes after a while and becomes incorrect. The Backup-counter is correct, though.

To Reproduce
Steps to reproduce the behavior:

1. Uninstall app
2. Install app
3. Login with the same account
4. Select some photos which are already uploaded

Expected behavior
Immich should detect that the photos already exist and shouldn't try to upload them.

Screenshots

System

• Phone OS [Android]: 12
• Server Version: 1.44.0
• Mobile App Version: 1.44.0

Additional context
Add any other context about the problem here.

• I use oauth2 for authentication
  docker-compose.yml

version: "3.8"

services:
  immich-server:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    image: itsblue.dev/dorian/immich-machine-learning-no-avx
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - database
    restart: always

  immich-web:
    image: altran1502/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always
    labels:
      traefik.enable: true
      traefik.http.routers.immich.rule: "Host(`photos.example.com`)"
      traefik.http.routers.immich.entrypoints: webs

.env

DB_HOSTNAME=immich_postgres
DB_USERNAME=<user>
DB_PASSWORD=<password>
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

UPLOAD_LOCATION=/mnt/data/immich

JWT_SECRET=<secret>

MAPBOX_KEY
ENABLE_MAPBOX=true
MAPBOX_KEY=<key>

VITE_LOGIN_PAGE_MESSAGE=

[alextran1502]

The device id is indicated by how the app is installed. So by switching from fdroid to apk, it would have a different devive id, lead to the server treat this as a new device and thus the reason it would reupload everything.

[dorianim]

Ah, I see.
Maybe, it could be resolved by sending all hashes of local images, before trying to upload them, and only uploading the images with unknown hashes?

[mike-lloyd03]

Oh wow this is exactly the issue we were just discussing on Discord.

I solved this pretty easily by updating the assets in the database to be linked to your new deviceId

Here's how I did it:

connect to the database with docker compose exec:

docker compose exec -it database psql -U postgres -d immich

Get the new and old deviceIds:

SELECT "deviceId", "createdAt" FROM device_info;

                             deviceId                             |         createdAt
------------------------------------------------------------------+----------------------------
 afe72a231ad8c4e0e95d60b89a003eb5afe0b2063b918c4f555062fcfe8bffca | 2023-01-22 03:06:08.953061
 d6744e110081150d480a99e1b1f815d4ddf34d957b2ee17d0b59a498964516ce | 2023-02-02 17:20:02.907193

The createdAt dates should help identify which is the older device. Copy both deviceIds. Then run the following query:

UPDATE assets SET "deviceId" = 'newId' WHERE "deviceId" = 'oldId';

Pay attention to single and double quotes as these matter in postgres. Replace newId and oldId with what you copied earlier.

You can then exit the database shell with \q.

I had to force close the app on my phone and reopen it but that fixed my issue. You may need to delete local storage on your phone and log back in if that doesn't work. But after doing this, all the existing assets on my phone were correctly identified with the assets already uploaded on the server.

[Snuupy]

fyi if you're rooted, you can run corepatch and update from either source and have it ignore signature checks.

[alextran1502]

Closing as this is the current expecting behavior when switching installation method

[dorianim]

@alextran1502 I think, this should be fixed nevertheless...
It makes the app unusable.

@mike-lloyd03 thanks for the workaround! It stopped uploading the old assets, but now the remainder counter is negative...

[alextran1502]

@dorianim this is not the problem with the app I would argue, it is how the id is generated. So I would either stick with one the PlayStore, FDroid or apk installation method and not switching between those

[dorianim]

don't you think, that switching of the apps source is a scenario that should be supported?

I have the same problem in the CLI since I switched from username and password to api tokens.
Wouldn't it be cleaner to handle this without binding the assets to a device?

[alextran1502]

@dorianim That is the plan to eventually move away from checking device id. So we will focus on doing that instead of working out a solution for this scenario

[dorianim]

Yes, that would be great! Is there an issue to keep track of that?

[mike-lloyd03]

@dorianim clearing the app's local storage and logging back in should fix the negative counter issue. I had the same problem.

[jrasm91]

#1553 and #731

[Snuupy]

I was having some trouble figuring out which device had which photos, here are some diagnostic postgres commands to run to figure out which deviceId is which by looking at the number of assets.

You can figure out userId based on the immich data folder

You can figure out the set of deviceIds by doing a SELECT * FROM device_info WHERE "userId" = "YOUR_USER_ID";

Then you can look at the count of assets per device ID: SELECT COUNT(*) FROM assets WHERE "deviceId" = 'deviceId';

Hopefully that helps you identify which device is which.

[DX37]

Can't find device_info table, but found assets with deviceId column... How can I find new deviceId?

[DX37]

Is this behaviour is still expected? I got another ROM for my Android device and Immich showing assets like uploaded in feed, but in backup page it still wants to upload them all. If it uploads them, in app it marks them as Duplicated Assets, and after CLEAR it wants to upload them all again...

Perhaps I can change ownerId again in database, but... Should I?

[alextran1502]

@DX37 still the same owner right? Just different device

[DX37]

@DX37 still the same owner right? Just different device

Well, theoretically speaking, yes - different device (another ROM, installed crDroid instead of Pixel Experience), but still same email and password.

[DX37]

Just like in this comment, but on Android #1553 (comment)

[alextran1502]

@DX37 So you can change the deviceId, not the ownerId

[DX37]

@DX37 So you can change the deviceId, not the ownerId

Oh, right, I mistyped. I meant deviceId of course.

Well I changed them all. App shows a little amount of uploaded images now, around 10%, but still requests to upload.
I have total amount of 1475 assets. Before change there was 0 uploaded and 1475 remaining. After change to ALL assets in library there's 124 uploaded and 1351 remaining... -__-

These 1475 assets was ALL uploaded before factory reset and firmware change.

[DX37]

Hm... This what happens when asset wants to upload anyway

immich_postgres          | 2023-11-19 08:07:00.344 UTC [1118] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
immich_postgres          | 2023-11-19 08:07:00.344 UTC [1118] DETAIL:  Key ("ownerId", "libraryId", checksum)=(a6342db4-ff01-4a77-99ab-fd45f69bca6f, 2d8042eb-b1ed-4220-b3fe-34f6fad5eb33, \xc9aa63a73396a6e6afde0b333a98a0f073771733) already exists.
immich_postgres          | 2023-11-19 08:07:00.344 UTC [1118] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackParentId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, DEFAULT, DEFAULT, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, DEFAULT, $23, $24, DEFAULT) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "isVisible"

I tried to "upload" all remaining assets, and in the end it was 124 total, 124 uploaded, 0 remaining and 1351 Duplicated Assets, which after CLEAR reverted to numbers in my previous message.