Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The learning rate detected in the optimizer is not a Parameter type, so it is not recorded. Its type is '_IteratorLearningRate'. #677

Open
TanateT opened this issue Mar 6, 2024 · 3 comments

Comments

@TanateT
Copy link

TanateT commented Mar 6, 2024

Hello author, when I was using SummaryCollector for visualization, I encountered the following error:
The learning rate detected in the optimizer is not a Parameter type, so it is not recorded It type is'InteratorLearningRate '.
May I ask what is causing this?

@panshaowu
Copy link
Collaborator

@TanateT
Hello, thanks for your feedback.
It seems that the problem was caused by an improper definition of the learning rate.
You can try to debug the code on PyNative mode and to check the data type of the learning rate input to the optimizer.
If you need further help, please provide more details on your code.

@TanateT
Copy link
Author

TanateT commented Mar 8, 2024

@panshaowu ,
Hello,When I use the following code to train the icdar_15 dataset using CRNN and SVTR, the above error will appear. You can try it on the GPU.
summary=SummaryCollector (summarid_dir='summary', collect_freq=100)

@panshaowu
Copy link
Collaborator

panshaowu commented Mar 12, 2024

@TanateT Hello, I tried to modify the python script tools/train.py (refer to Tutorials):

def main(cfg):
    ....
    # training
    model = ms.Model(train_net)
    summary_collector = SummaryCollector(summary_dir='summary_dir', collect_freq=100)
    model.train(
        cfg.scheduler.num_epochs,
        loader_train,
        callbacks=[eval_cb, summary_collector],
        dataset_sink_mode=cfg.train.dataset_sink_mode,
        initial_epoch=start_epoch,
    )

And then I tested the function of SummaryCollector on a GPU server (1080Ti, CUDA 10.1, MindSpore 2.2.11) using following commands:

clear && python tools/train.py -c configs/rec/crnn/crnn_icdar15.yaml

No errors were raised. And SummaryCollector worked normally. I could see the captured training info using MindInsight.
I am not sure whether you added other customized codes which might cause problems.
Could you please provide more details?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants