Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always stuck in step5 [5/8 - Executing task CreateClustersTask (Create new persons or update existing persons)] #712

Open
3038922 opened this issue Dec 2, 2023 · 3 comments

Comments

@3038922
Copy link

3038922 commented Dec 2, 2023

I use external decoding and then he always gets stuck.Why is that?And my nextcloud's memory keeps taking up until it's full.My decoding VM sets up model 3.

 ⚡ root@nextcloud  ~  sudo -u www-data php /var/www/nextcloud/occ face:background_job
1/8 - Executing task CheckRequirementsTask (Check all requirements)
2/8 - Executing task CheckCronTask (Check that service is started from either cron or from command)
3/8 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.)
4/8 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB)
5/8 - Executing task CreateClustersTask (Create new persons or update existing persons)
        Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
        * have 1000 faces already processed
        * or you need to have 95% of you images processed
        Use stats command to track progress
        Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
        * have 1000 faces already processed
        * or you need to have 95% of you images processed
        Use stats command to track progress
        Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
        * have 1000 faces already processed
        * or you need to have 95% of you images processed
        Use stats command to track progress
        Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
        * have 1000 faces already processed
        * or you need to have 95% of you images processed
        Use stats command to track progress
        Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
        * have 1000 faces already processed
        * or you need to have 95% of you images processed
        Use stats command to track progress
        Face clustering will be created for the first time.
        0 faces found for clustering
        0 clusters found after clustering
        Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
        * have 1000 faces already processed
        * or you need to have 95% of you images processed
        Use stats command to track progress
        Face clustering will be created for the first time.
@matiasdelellis
Copy link
Owner

matiasdelellis commented Jan 9, 2024

Hi @3038922
I understand that you have many images analyzed, and many faces to clustering.. It is strange that you run out of resources, but it is a very demanding task. Can you use the face:stats command to contextualize?

If you are really running out of ram, you can add SWAP, and increase the php_limits

p.s: sorry for delay. I'm very busy. 😞

@vwbusguy
Copy link

vwbusguy commented Jun 3, 2024

I ran into this recently and it just took a very long time, but eventually made it through step 5. I had to run it outside of cron without the timeout since it was timing out on this step within 20 minutes before working on faces.

matiasdelellis added a commit that referenced this issue Jun 4, 2024
This improves the performance of clustering with many faces. In my
tests with 43 thousand faces, it takes 11,91 minutes to run the full
'face:background_job --cluster-mode' command. Using a batch size of 20
thousand the time is reduced to 4.47 minutes. With 5000, 1.71 minute
and with 2000 (Which is the minimum cut it takes only 54 seconds.

Against all odds, memory consumption does not increase in any way,
but it still has a disadvantage. The clusters... although they do not
increase their quantity as much (only 5%), these will generally be
smaller (Presumably of higher quality), but there will be more
clusters to give them names.

Well, this is another advanced option that will not be available in
the administrator interface.

occ config:app:set  facerecognition clustering_batch_size --value='1000' --type=integer
@matiasdelellis
Copy link
Owner

Hi both,
You can try applying the latest commit to your installation. They should help you a lot.

58e3e0e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants