i was made classification code before. But now that code isn't working. How should I change it?

[sinyoung-park]

i use colab and this is my config file .

%%writefile configs/resnet/resnet34_8xb32_in1k_custom.py
base = [
'../base/models/resnet34.py', '../base/datasets/imagenet_bs32.py',
'../base/schedules/imagenet_bs256.py', '../base/default_runtime.py'
]

model settings

model = dict(
type='ImageClassifier',
backbone=dict(
type='ResNet',
depth=34,
num_stages=4,
out_indices=(3, ),
style='pytorch'),
neck=dict(type='GlobalAveragePooling'),
head=dict(
type='LinearClsHead',
num_classes=5,
in_channels=512,
loss=dict(type='CrossEntropyLoss', loss_weight=0.8),
topk=(1, 5),
))

dataset settings

dataset_type = 'CustomDataset'
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='RandomResizedCrop', size=224),
dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
dict(type='Normalize', **img_norm_cfg),
dict(type='ImageToTensor', keys=['img']),
dict(type='ToTensor', keys=['gt_label']),
dict(type='Collect', keys=['img', 'gt_label'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1)),
dict(type='CenterCrop', crop_size=224),
dict(type='Normalize', **img_norm_cfg),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
]
data = dict(
samples_per_gpu=16,
workers_per_gpu=2,
train=dict(
type=dataset_type,
data_prefix='/content/mmclassification/pcldata/train_set',
ann_file='/content/mmclassification/pcldata/training.txt',
classes = ['8slm', '12slm', '16slm', '20slm', '24slm'],
pipeline=train_pipeline),
val=dict(
type=dataset_type,
data_prefix='/content/mmclassification/pcldata/val_set',
ann_file='/content/mmclassification/pcldata/validation.txt',
classes = ['8slm', '12slm', '16slm', '20slm', '24slm'],
pipeline=test_pipeline),
test=dict(
# replace data/val with data/test for standard test
type=dataset_type,
data_prefix='/content/mmclassification/pcldata/test_set',
ann_file='/content/mmclassification/pcldata/test.txt',
classes = ['8slm', '12slm', '16slm', '20slm', '24slm'],
pipeline=test_pipeline))
evaluation = dict(interval=1, metric='accuracy')

optimizer

optimizer = dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)

learning policy

lr_config = dict(policy='step', step=[30, 60, 90])
runner = dict(type='EpochBasedRunner', max_epochs=12)

checkpoint saving

checkpoint_config = dict(interval=1)

yapf:disable

log_config = dict(
interval=100,
hooks=[
dict(type='TextLoggerHook'),
# dict(type='TensorboardLoggerHook')
])

yapf:enable

dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]

i modifiy this code. and i train this model, but the code isn't working.

!python tools/train.py /content/mmclassification/configs/resnet/resnet34_8xb32_in1k_custom.py --work-dir /resnet34_8xb32_in1k_custom

04/10 02:19:23 - mmengine - INFO -

System environment:
sys.platform: linux
Python: 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0]
CUDA available: True
numpy_random_seed: 755549834
GPU 0: Tesla T4
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.8, V11.8.89
GCC: x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
PyTorch: 2.0.0+cu118
PyTorch compiling details: PyTorch built with:

• GCC 9.3

• C++ Version: 201703

• Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications

• Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)

• OpenMP 201511 (a.k.a. OpenMP 4.5)

• LAPACK is enabled (usually provided by MKL)

• NNPACK is enabled

• CPU capability usage: AVX2

• CUDA Runtime 11.8

• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90

• CuDNN 8.7

• Magma 2.6.1

• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

  TorchVision: 0.15.1+cu118
  OpenCV: 4.7.0
  MMEngine: 0.7.2

Runtime environment:
cudnn_benchmark: False
mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}
dist_cfg: {'backend': 'nccl'}
seed: None
deterministic: False
Distributed launcher: none
Distributed training: False
GPU number: 1

04/10 02:19:23 - mmengine - INFO - Config:
model = dict(
type='ImageClassifier',
backbone=dict(
type='ResNet',
depth=34,
num_stages=4,
out_indices=(3, ),
style='pytorch'),
neck=dict(type='GlobalAveragePooling'),
head=dict(
type='LinearClsHead',
num_classes=5,
in_channels=512,
loss=dict(type='CrossEntropyLoss', loss_weight=0.8),
topk=(1, 5)))
dataset_type = 'CustomDataset'
data_preprocessor = dict(
num_classes=1000,
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='RandomResizedCrop', size=224),
dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='ToTensor', keys=['gt_label']),
dict(type='Collect', keys=['img', 'gt_label'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1)),
dict(type='CenterCrop', crop_size=224),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
]
train_dataloader = dict(
pin_memory=True,
persistent_workers=False,
collate_fn=dict(type='default_collate'),
batch_size=32,
num_workers=5,
dataset=dict(
type='ImageNet',
data_root='data/imagenet',
ann_file='meta/train.txt',
data_prefix='train',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='RandomResizedCrop', scale=224),
dict(type='RandomFlip', prob=0.5, direction='horizontal'),
dict(type='PackInputs')
]),
sampler=dict(type='DefaultSampler', shuffle=True))
val_dataloader = dict(
pin_memory=True,
persistent_workers=False,
collate_fn=dict(type='default_collate'),
batch_size=32,
num_workers=5,
dataset=dict(
type='ImageNet',
data_root='data/imagenet',
ann_file='meta/val.txt',
data_prefix='val',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='ResizeEdge', scale=256, edge='short'),
dict(type='CenterCrop', crop_size=224),
dict(type='PackInputs')
]),
sampler=dict(type='DefaultSampler', shuffle=False))
val_evaluator = dict(type='Accuracy', topk=(1, 5))
test_dataloader = dict(
pin_memory=True,
persistent_workers=False,
collate_fn=dict(type='default_collate'),
batch_size=32,
num_workers=5,
dataset=dict(
type='ImageNet',
data_root='data/imagenet',
ann_file='meta/val.txt',
data_prefix='val',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='ResizeEdge', scale=256, edge='short'),
dict(type='CenterCrop', crop_size=224),
dict(type='PackInputs')
]),
sampler=dict(type='DefaultSampler', shuffle=False))
test_evaluator = dict(type='Accuracy', topk=(1, 5))
optim_wrapper = dict(
optimizer=dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001))
param_scheduler = dict(
type='MultiStepLR', by_epoch=True, milestones=[30, 60, 90], gamma=0.1)
train_cfg = dict(by_epoch=True, max_epochs=100, val_interval=1)
val_cfg = dict()
test_cfg = dict()
auto_scale_lr = dict(base_batch_size=256)
default_scope = 'mmpretrain'
default_hooks = dict(
timer=dict(type='IterTimerHook'),
logger=dict(type='LoggerHook', interval=100),
param_scheduler=dict(type='ParamSchedulerHook'),
checkpoint=dict(type='CheckpointHook', interval=1),
sampler_seed=dict(type='DistSamplerSeedHook'),
visualization=dict(type='VisualizationHook', enable=False))
env_cfg = dict(
cudnn_benchmark=False,
mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
dist_cfg=dict(backend='nccl'))
vis_backends = [dict(type='LocalVisBackend')]
visualizer = dict(
type='UniversalVisualizer', vis_backends=[dict(type='LocalVisBackend')])
log_level = 'INFO'
load_from = None
resume = False
randomness = dict(seed=None, deterministic=False)
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
data = dict(
samples_per_gpu=16,
workers_per_gpu=2,
train=dict(
type='CustomDataset',
data_prefix='/content/mmclassification/pcldata/train_set',
ann_file='/content/mmclassification/pcldata/training.txt',
classes=['8slm', '12slm', '16slm', '20slm', '24slm'],
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='RandomResizedCrop', size=224),
dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='ToTensor', keys=['gt_label']),
dict(type='Collect', keys=['img', 'gt_label'])
]),
val=dict(
type='CustomDataset',
data_prefix='/content/mmclassification/pcldata/val_set',
ann_file='/content/mmclassification/pcldata/validation.txt',
classes=['8slm', '12slm', '16slm', '20slm', '24slm'],
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1)),
dict(type='CenterCrop', crop_size=224),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
]),
test=dict(
type='CustomDataset',
data_prefix='/content/mmclassification/pcldata/test_set',
ann_file='/content/mmclassification/pcldata/test.txt',
classes=['8slm', '12slm', '16slm', '20slm', '24slm'],
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1)),
dict(type='CenterCrop', crop_size=224),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
]))
evaluation = dict(interval=1, metric='accuracy')
optimizer = dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(policy='step', step=[30, 60, 90])
runner = dict(type='EpochBasedRunner', max_epochs=12)
checkpoint_config = dict(interval=1)
log_config = dict(interval=100, hooks=[dict(type='TextLoggerHook')])
dist_params = dict(backend='nccl')
resume_from = None
workflow = [('train', 1)]
launcher = 'none'
work_dir = '/resnet34_8xb32_in1k_custom'

04/10 02:19:29 - mmengine - INFO - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.
04/10 02:19:29 - mmengine - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH ) RuntimeInfoHook
(BELOW_NORMAL) LoggerHook

before_train:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(VERY_LOW ) CheckpointHook

before_train_epoch:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(NORMAL ) DistSamplerSeedHook

before_train_iter:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook

after_train_iter:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

after_train_epoch:
(NORMAL ) IterTimerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

before_val_epoch:
(NORMAL ) IterTimerHook

before_val_iter:
(NORMAL ) IterTimerHook

after_val_iter:
(NORMAL ) IterTimerHook
(NORMAL ) VisualizationHook
(BELOW_NORMAL) LoggerHook

after_val_epoch:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

after_train:
(VERY_LOW ) CheckpointHook

before_test_epoch:
(NORMAL ) IterTimerHook

before_test_iter:
(NORMAL ) IterTimerHook

after_test_iter:
(NORMAL ) IterTimerHook
(NORMAL ) VisualizationHook
(BELOW_NORMAL) LoggerHook

after_test_epoch:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook

after_run:
(BELOW_NORMAL) LoggerHook

Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/content/mmclassification/mmpretrain/datasets/imagenet.py", line 39, in init
super().init(
File "/content/mmclassification/mmpretrain/datasets/custom.py", line 219, in init
self.full_init()
File "/content/mmclassification/mmpretrain/datasets/base_dataset.py", line 176, in full_init
super().full_init()
File "/usr/local/lib/python3.9/dist-packages/mmengine/dataset/base_dataset.py", line 301, in full_init
self.data_list = self.load_data_list()
File "/content/mmclassification/mmpretrain/datasets/custom.py", line 266, in load_data_list
lines = list_from_file(self.ann_file)
File "/usr/local/lib/python3.9/dist-packages/mmengine/fileio/parse.py", line 60, in list_from_file
text = get_text(filename, encoding, backend_args=backend_args)
File "/usr/local/lib/python3.9/dist-packages/mmengine/fileio/io.py", line 208, in get_text
return backend.get_text(filepath, encoding)
File "/usr/local/lib/python3.9/dist-packages/mmengine/fileio/backends/local_backend.py", line 56, in get_text
with open(filepath, encoding=encoding) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/meta/train.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/content/mmclassification/tools/train.py", line 159, in
main()
File "/content/mmclassification/tools/train.py", line 155, in main
runner.train()
File "/usr/local/lib/python3.9/dist-packages/mmengine/runner/runner.py", line 1672, in train
self._train_loop = self.build_train_loop(
File "/usr/local/lib/python3.9/dist-packages/mmengine/runner/runner.py", line 1471, in build_train_loop
loop = EpochBasedTrainLoop(
File "/usr/local/lib/python3.9/dist-packages/mmengine/runner/loops.py", line 44, in init
super().init(runner, dataloader)
File "/usr/local/lib/python3.9/dist-packages/mmengine/runner/base_loop.py", line 26, in init
self.dataloader = runner.build_dataloader(
File "/usr/local/lib/python3.9/dist-packages/mmengine/runner/runner.py", line 1346, in build_dataloader
dataset = DATASETS.build(dataset_cfg)
File "/usr/local/lib/python3.9/dist-packages/mmengine/registry/registry.py", line 545, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/usr/local/lib/python3.9/dist-packages/mmengine/registry/build_functions.py", line 135, in build_from_cfg
raise type(e)(
FileNotFoundError: class ImageNet in mmpretrain/datasets/imagenet.py: [Errno 2] No such file or directory: 'data/imagenet/meta/train.txt'

i use customdataset.
how can i fix it?

[Ezra-Yu]

I'm sorry for the inconvenience of this upgrade, but if you were using version 0.x of the code, here is the migration documentation