PaperWithCode eight data set model ranking: https://paperswithcode.com/task/crowd-counting
search keyword
"Crowd counting" (crowd counting, crowd refers to people rather than crowded people; Counting People ) ; Passenger flow statistics", "Person counter", "Person counter"; crowd density
"Real-time pedestrian analysis", Crowd Counting+Localization=Crowd Analysis; analyze some characteristics of pedestrians Pedestrian Attribute Recognition
Technical implementation: object detection, object tracking , deduplication
(Universal Code) CommissarMa/Crowd_counting_from_scratch
https://github.com/CommissarMa/Crowd_counting_from_scratch
加载数据集需要用到Torch，但我我的ubuntu电脑没有装Torch，我不知道转哪个版本的ubuntu，方便后面的十几个模型的使用
(S1) Load the data set and make the ground truth of the density map (the ground truth of the data set in shanghai tech does not require you to make it, it is provided by the official)
The code it provides is without the codes of train.py and test.py
dataloader_example.py provides the code for dataset loading
fdst_densitymap_prepare.py provides data loading and ground truth production for the fsdt dataset
Three kinds of Gaussian kernels to make the density map of ground truth "k_nearest_gaussian_kernel.py", "same_gaussian_kernel.py"
(S2) The data set and ground truth are fed into the model designed by the author, and the training starts
The /crowd_model/mcnn_model.py file provides the network structure, forward rebroadcast, and the final prediction value. but without backpropagation
Code without backpropagation and parameter update
(S3) Take the trained model, feed in the pictures of the test set, output a density map, sum the integrals, compare the number of people with the ground truth, and get the loss value score
(Universal code) C3F
Understand the idea and logic of the C3F code (see his Zhihu blog), try to run the code : https://zhuanlan.zhihu.com/p/65650998
Popular science Chinese blog post: https://zhuanlan.zhihu.com/p/65650998
Framework URL: https://github.com/gjy3035/C-3-Framework
Web-side crowd counting labeling tool: https://github.com/Elin24/cclabeler
The first step - configure the environmentPython 3.xPytorch1.0 (some networks only support 0.4):http://pytorch.org. (Some networks can only use torchversion 0.4)otherlibsin requirements.txt, run pipinstall- r requirements.txt.torch, torchvision, tensorboardX, tensorboard, tensorflow, easydict,pandas,numpy, scipyThe second step is to download the dataset and change the location of the dataset in the code to the location where I saved the dataset. Run the code to generate a density map. Where is the dataset you mentioned downloaded on my computer: /media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/open_datasetWrite this location into the code that generates the density map
For the filling method, refer to my modified file——.\C-3-Framework\datasets\SHHA\preapre_SHHA.m
preapre_SHHA.m (get it done)——F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\C-3-Framework\datasets\data\768x1024RGB-k15-s4\shanghaitech_part_A
preapre_SHHB.m (get it done)——F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\C-3-Framework\datasets\data\768x1024RGB-k15-s4\shanghaitech_part_B
prepare_GCC.m——(not done) GCC data set, a data set of 3-6 G, too big, and finally use it again
preapre_QNRF.m - (get it)——test——F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\C-3-Framework\datasets\UCF-qnrf\1024x1024_mod16\test
——train——F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\C-3-Framework\datasets\UCF-qnrf-processed\train
preapre_UCF_CC_50.m (fixed)——In addition to the generation of density map, this data set also has segmentation
——His MATLAB code requires that your data set must provide ann.mat, but the data set I got has many scene scences, each scene has many videos, and there is no ann.mat annotation file— —Look at the one drive network disk provided by the author and download the data set provided by the author
——Fixed, but did not run the .m program, did not find this data set, and directly downloaded the ground truth made by the author
——F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\C-3-Framework\datasets\UCF\UCF_CC_50
preapre_WE.m (get it done)——WE=World Expo'10 data set, I did not download this data set, and directly used the ground truth provided by the author
Put these processed ground truth folders into this folder
/ProcessedData. The folder structure is as follows. The name of each dataset folder should also be changed to the following
+-- C-3-Framework | +-- datasets | +-- misc | +-- ...... +-- ProcessedData | +-- shanghaitech_part_A | +-- shanghaitech_part_B | +-- UCF_CC_50 | +-- UCF-QNRF-1024x1024-mod16 | +-- WE_blurred
Pretrained Model
Some Counting Networks (such as VGG, CSRNet and so on) adopt the pre-trained models on ImageNet. You can download them from TorchVision
Place the processed model to
~/.cache/torch/checkpoints/(only for linux OS).
This location also exists in win10, (C:\Users\hasee\.cache\torch\hub\checkpoints), but the URL he provided above https://github.com/pytorch/vision/tree/main/torchvision/ The models are the .py code of the network. There are only network results and no parameters of the pre-trained model. Is it necessary to download it?
Another virtual environment (r-reticulate) of win10 requires me to put the model file in this location (C:\Users\hasee\.torch\models)
Download this model file and put it in the win10 address https://download.pytorch.org/models/resnet101-5d3b4d8f.pth
Training
set the parameters in
config.pyand./datasets/XXX/setting.py(if you want to reproduce our results, you are recommonded to use our parameters in./results_reports).run
python train.py.The error is as follows
My guess is that other people's GPUs have more blocks, and my GPU has fewer blocks (only 1 block). People use four GPUs to train models in parallel, but mine can’t
Traceback (most recent call last): File "train.py", line 62, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 25, in __init__ self.net = CrowdCounter(cfg.GPU_ID,self.net_name).cuda() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\models\CC.py", line 29, in __init__ self.CCN = torch.nn.DataParallel(self.CCN, device_ids=gpus).cuda() File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\nn\parallel\data_parallel.py", line 142, in __init__ _check_balance(self.device_ids) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\nn\parallel\data_parallel.py", line 23, in _check_balance dev_props = _get_devices_properties(device_ids) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\_utils.py", line 458, in _get_devices_properties return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids] File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\_utils.py", line 458, in <listcomp> return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids] File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\_utils.py", line 441, in _get_device_attr return get_member(torch.cuda) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\_utils.py", line 458, in <lambda> return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids] File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\cuda\__init__.py", line 299, in get_device_properties raise AssertionError("Invalid device id") AssertionError: Invalid device idIn this folder, change line31 in F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\config.py to the following and change from two GPUs to one
#__C.GPU_ID = [0,1] __C.GPU_ID = [0]Continue to run after the change, and the following error occurs-I suspect that the version of pytorch is too high, which caused some function interfaces in the past to be unusable now, so the error is reported
D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\optim\lr_scheduler.py:136: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html(Try1): Using the official code, try to install
pytorch1.0.0, (released on 8 December 2018) conda install pytorch==1.0.0 torchvision==0.2.1 -c pytorch (network problem, can’t download it for a long time, This package is not available)————The compression package directly downloaded from the official website is faster https://anaconda.org/pytorch/pytorch/files?version=1.0.0ButThe name of the virtual environment: (r-reticulate), Python version 3.6.13
torchvision=0.2.2whl file download address https://pypi.org/project/torchvision/0.2.2/#files
This command installation - pip install "file path" (don't
conda install torchvision=0。2。2 -c pytorchuse it for installation, because this code will automatically install your pytorch to the latest version 1.10.1, and your 1.0.0 pytorch will be destroyed )The version adopts the latest version before Jul 26, 2019
tensorboardX
Find the previous version of pytorch1.0.0 (released on 8 December 2018), use tensorboardX==1.4, (Released: Aug 9, 2018)
tensorboard (not installed)
tensorflow (not installed)
easydict=1.9————pip install easydict==1.9
pip install 文件名.tar.gzOr unzip, there is a setup.py file in the directory
Switch to this directory (cmd or enter cmd directly in the opened folder path (above))
Execute the command python setup.py install
pandas==0.24.2, Mar 14,2019pip install pandas==0.24.2
numpy== recommend this, when actually installing pandas, nump also installed 1.16.0, Jan 14, 2019Actually this version '1.19.5'i
scipy==1.2.1，Feb 9, 2019The following error appears again, it seems to be an error related to reading and saving files
——Read the blog of the framework to see other people's design ideas
——I suspect that shutil.copytree(file, dst_file) wants to copy this to a place that already has this folder, which can be solved by modifying the parameters in the function or adding an if to do a filter
The official document of the shutil.copytree(file, dst_file) function has the following passage, telling us that if the parameter dirs_exist_ok uses the default false, then if the folder copied to the location already exists, no error will be reported
https://docs.python.org/3/library/shutil.html
If dirs_exist_ok is false (the default) and dst already exists, a FileExistsError is raised. If dirs_exist_ok is true, the copying operation will continue if it encounters existing directories, and files within the dst tree will be overwritten by corresponding files from the src tree.
But only shutil.copytree() in this function of Python3.11.1 has this parameter dirs_exist_ok, I am currently using version 3.6.13 without this parameter
(1) Check how the parameter is written in the 3.11 version of python. If there are not too many related and dependent functions, I will imitate it and add it to the code of my system.
I backed up the original "shutil.py" file of Python=3.6.13 and put it on the desktop
Modify all functions related tocopytree() according to Python3.11.0
(2) See how people add this parameter. Is it an if else judgment or a try exception?
(3) In fact, the shutil.copytree(file, dst_file) function cannot add this function. You can use try exption or if else to skip this error]
One possibility is that you are not Linux, and the slashes in the address of the file look different from win10
——Go to Linux and configure a virtual environment with the same version number, try it, if it doesn’t work,find
Another possibility is that you need to add an if judgment, if the file exists, it will no longer be created
Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 62, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 52, in __init__ self.writer, self.log_txt = logger(self.exp_path, self.exp_name, self.pwd, 'exp', resume=cfg.RESUME) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\misc\utils.py", line 76, in logger copy_cur_env(work_dir, exp_path+ '/' + exp_name + '/code', exception) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\misc\utils.py", line 249, in copy_cur_env shutil.copytree(file, dst_file) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\shutil.py", line 321, in copytree os.makedirs(dst) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\os.py", line 220, in makedirs mkdir(name, mode) FileExistsError: [WinError 183] 当文件已存在时，无法创建该文件。: './exp/12-08_16-20_SHHB_Res101_SFCN_1e-05/code\\.git'Then continue to run and encounter the following error
File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\shutil.py", line 343, in _copytree raise Error(errors) shutil.Error: [('F:\\FILES_OF_ALBERT\\IT_paid_class\\graduation_thesis\\model_innov\\2021_C-3-Framework\\C-3-Framework\\.git\\objects\\pack\\pack-f045ed913ad90b5201f2b6628b5474cac03e5607.idx', './exp/12-08_20-16_SHHB_Res101_SFCN_1e-05/code\\.git\\objects\\pack\\pack-f045ed913ad90b5201f2b6628b5474cac03e5607.idx', "[Errno 13] Permission denied: './exp/12-08_20-16_SHHB_Res101_SFCN_1e-05/code\\\\.git\\\\objects\\\\pack\\\\pack-f045ed913ad90b5201f2b6628b5474cac03e5607.idx'"), ('F:\\FILES_OF_ALBERT\\IT_paid_class\\graduation_thesis\\model_innov\\2021_C-3-Framework\\C-3-Framework\\.git\\objects\\pack\\pack-f045ed913ad90b5201f2b6628b5474cac03e5607.pack', './exp/12-08_20-16_SHHB_Res101_SFCN_1e-05/code\\.git\\objects\\pack\\pack-f045ed913ad90b5201f2b6628b5474cac03e5607.pack', "[Errno 13] Permission denied: './exp/12-08_20-16_SHHB_Res101_SFCN_1e-05/code\\\\.git\\\\objects\\\\pack\\\\pack-f045ed913ad90b5201f2b6628b5474cac03e5607.pack'")]To open a cmd with administrator privileges to run this code
Either remove this read-only in the properties of the folder,
After the above error is fixed, continue to run and encounter an error that cuda memory is not enough
Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 64, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 26, in __init__ self.net = CrowdCounter(cfg.GPU_ID,self.net_name).cuda() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\models\CC.py", line 31, in __init__ fatal fatal self.CCN=self.CCN.cuda() : File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\nn\modules\module.py", line 260, in cuda : Memory allocation failureMemory allocation failure return self._apply(lambda t: t.cuda(device)) Traceback (most recent call last): File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\nn\modules\module.py", line 187, in _apply Traceback (most recent call last): File "<string>", line 1, in <module> module._apply(fn) File "<string>", line 1, in <module> fatal File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\nn\modules\module.py", line 187, in _apply File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main : exitcode = _main(fd) module._apply(fn) exitcode = _main(fd) Memory allocation failure File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\nn\modules\module.py", line 193, in _apply File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) param.data = fn(param.data) prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare Traceback (most recent call last): File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\nn\modules\module.py", line 260, in <lambda> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "<string>", line 1, in <module> return self._apply(lambda t: t.cuda(device)) _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main RuntimeError: CUDA error: out of memory——Minimize the amount of data read in each batch of the code—see if it will exceed the memory? - still will exceed
——Try it, it seems that only one picture is read in each batch (I didn’t see the description about the batch size, only __C.VISIBLE_NUM_IMGS = 1 in line 71 of config.py seems to read one picture at a time) , but only one picture + various calculations - the 4G video memory is not enough
——I read train.py, there is forward propagation and backpropagation (in trainer.py, def __init__() of class Trainer() is useful for Adam optimization, which is backpropagation), but train.py then only uses Trainer().forward()--but the forward function uses all the functions in class Trainer(), including train() validate_V1(), without model saving--is this really training?
——What data set ShanghaiTech_partB and model-Res101_SFCN are used?
——At present, it is suspected that it is the location of the data set. Dozens of photos are read in at a time as the data read in as a batch. Because if only one picture is read in at a time, the size of a picture in SHHBz is about 160KB, the size of the csv file of the density map is about 2MB, and the size of the pth file of the pre-training model parameters of resnet101 is 170MB, plus miscellaneous Computing data should not be able to take up more than 4GB of data- Where is this code used - line 27 of train.py is quoted
from datasets.SHHB.loading_data import loading_data from datasets.SHHB.setting import cfg_data——Where to change (line 20 of F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\datasets\SHHB\setting.py), change as follows, 6 pictures change to 1 picture .
# __C_SHHB.TRAIN_BATCH_SIZE = 6 #imgs __C_SHHB.TRAIN_BATCH_SIZE = 1 #imgs #__C_SHHB.VAL_BATCH_SIZE = 6 # __C_SHHB.VAL_BATCH_SIZE = 1 #report a new error
Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main Traceback (most recent call last): Traceback (most recent call last): exitcode = _main(fd) File "<string>", line 1, in <module> File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main prepare(preparation_data) exitcode = _main(fd) exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main _fixup_main_from_path(data['init_main_from_path']) prepare(preparation_data) prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) run_name="__mp_main__") _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> mod_name, mod_spec, pkg_name, script_name) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> cc_trainer.forward() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward cc_trainer.forward() exec(code, run_globals) self.train() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train self.train() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train cc_trainer.forward() for i, data in enumerate(self.train_loader, 0): for i, data in enumerate(self.train_loader, 0): File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ self.train() return _DataLoaderIter(self) return _DataLoaderIter(self) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ for i, data in enumerate(self.train_loader, 0): w.start() w.start() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) return _DataLoaderIter(self) self._popen = self._Popen(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) w.start() return _default_context.get_context().Process._Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) self._popen = self._Popen(self) return Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) return _default_context.get_context().Process._Popen(process_obj) prep_data = spawn.get_preparation_data(process_obj._name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main return Popen(process_obj) _check_not_importing_main() is not going to be frozen to produce an executable.''') File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main prep_data = spawn.get_preparation_data(process_obj._name) is not going to be frozen to produce an executable.''') File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. _check_not_importing_main() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> cc_trainer.forward() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward self.train() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train for i, data in enumerate(self.train_loader, 0): File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ return _DataLoaderIter(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ w.start() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> cc_trainer.forward() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward self.train() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train for i, data in enumerate(self.train_loader, 0): File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ return _DataLoaderIter(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ w.start() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> cc_trainer.forward() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward self.train() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train for i, data in enumerate(self.train_loader, 0): File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ return _DataLoaderIter(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ w.start() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. Traceback (most recent call last): Traceback (most recent call last): File "<string>", line 1, in <module> File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\runpy.py", line 85, in _run_code exec(code, run_globals) exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\train.py", line 65, in <module> cc_trainer.forward() cc_trainer.forward() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 66, in forward self.train() self.train() File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train File "F:\FILES_OF_ALBERT\IT_paid_class\graduation_thesis\model_innov\2021_C-3-Framework\C-3-Framework\trainer.py", line 87, in train for i, data in enumerate(self.train_loader, 0): for i, data in enumerate(self.train_loader, 0): File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__ return _DataLoaderIter(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ return _DataLoaderIter(self) w.start() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start w.start() self._popen = self._Popen(self) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\process.py", line 105, in start File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen self._popen = self._Popen(self) return _default_context.get_context().Process._Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 223, in _Popen File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen return _default_context.get_context().Process._Popen(process_obj) return Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\context.py", line 322, in _Popen File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ return Popen(process_obj) prep_data = spawn.get_preparation_data(process_obj._name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data prep_data = spawn.get_preparation_data(process_obj._name) File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() _check_not_importing_main() File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main File "D:\Programing_File\Anaconda3\envs\r-reticulate\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable.RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable.I generally read the error message of win10, and it is difficult to solve. I can't see what caused this error. ———I want to try running on linux to see if there will be this error. (There is a high probability that the code will run on linux in the future, so it is better to practice it first) ——If there is still this error on linux, I will find a way to solve it myself.
Configure the linux code environment
——Python installed
Python: When the code was written Jul 2, 2019, install pythonPython 3.4.10(released on 18 March 2019)https://www.python.org/downloads/
——The final python3.4.5 actually installed, because this version is on the anconda server, but the 3.5.10 one is not
conda create -n py34tc100 python=3.4--install pytorch=1.0.0
https://anaconda.org/pytorch/pytorch/files?version=1.0.0
The minimum version of python3.x of the linux system that matches pytorch1.0.0 is 3.5
current python version = 3.4.5
Check out the current cuda version = 10.1.168
How to query the version number of Cudann? ——I didn’t know that I was a match until I was queried
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2The version number of cudnn found out is 7.6.5
The pytorch=1.0.0 installation package that can be downloaded officially provided by the linux side
linux-64/pytorch-1.0.0-py3.7_cuda8.0.61_cudnn7.1.2_1.tar.bz2
linux-64/pytorch-1.0.0-py3.6_cuda8.0.61_cudnn7.1.2_1.tar.bz2
linux-64/pytorch-1.0.0-py3.5_cuda8.0.61_cudnn7.1.2_1.tar.bz2
linux-64/pytorch-1.0.0-py2.7_cuda8.0.61_cudnn7.1.2_1.tar.bz2| linux-64/pytorch-1.0.0-py3.7_cuda10.0.130_cudnn7.4.1_1.tar.bz2
linux-64/pytorch-1.0.0-py3.6_cuda10.0.130_cudnn7.4.1_1.tar.bz2
linux-64/pytorch-1.0.0-py3.5_cuda10.0.130_cudnn7.4.1_1.tar.bz2
linux-64/pytorch-1.0.0-py2.7_cuda10.0.130_cudnn7.4.1_1.tar.bz2
linux-64/pytorch-1.0.0-py2.7_cuda9.0.176_cudnn7.4.1_1.tar.bz2linux-64/pytorch-1.0.0-py3.5_cuda9.0.176_cudnn7.4.1_1.tar.bz2
linux-64/pytorch-1.0.0-py3.7_cuda9.0.176_cudnn7.4.1_1.tar.bz2
linux-64/pytorch-1.0.0-py3.6_cuda9.0.176_cudnn7.4.1_1.tar.bz2
Conclusion 3: What number should be installed in python, and the corresponding release date is the year, month, and date. Is it before the release date of the code?
When the CF3 code was released on GitHub: Jul 2, 2019
The python version required by pytorch1.0.0 requires a minimum of 3.5 (release date, 3.5.0= Sept. 13, 2015)
After which release time is the recommended installation of python? Most of the codes were released after 2018, it is recommended to install the ones released in 2017
Conclusion, install Python 3.6.0 Published: Dec. 23, 2016
https://www.python.org/downloads/
delete environment py34tc100
Open a virtual environment and install python=3.6.0
condacreate --name my_first_env python=3.6.0
InstallpytorchOfficial website command: https://pytorch.org/get-started/previous-versions/
conda install pytorch==1.0.0 torchvision==0.2.1 cuda100 -c pytorchThe package actually downloaded by this command is this pytorch-1.0.0-py3.6_cuda10.0.130_cudnn7.4.1_1
Install torchvision - already installed above
tensorboardX
Find the previous version of pytorch1.0.0 (released on 8 December 2018), use tensorboardX==1.4, (Released: Aug 9, 2018)
conda install tensorboardX==1.4easydict=1.9————
pip install easydict==1.9pandas ==0.24.2, Mar 14,2019
pip install pandas==0.24.2numpy == installed with pandas, the version is 1.19.2
scipy==1.2.1，Feb 9, 2019
pip install scipy==1.2.1The pre-trained model download is placed in the specified location
Put this and the training model (https://download.pytorch.org/models/resnet101-5d3b4d8f.pth) under this directory in the linux environment /home/albert/.torch/models/resnet101-5d3b4d8f.pth
run code
Modify the folder location: ——/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/SHHB/prepareSHHB——.m file does not need to be changedRunning reported the following error
/2021_C-3-Framework/C-3-Framework/misc/utils.py, line 253, in copy_cur_env shutil.copytree(file, dst_file, dirs_exist_ok=True), delete this parameter dirs_exist_ok
Traceback (most recent call last): File "train.py", line 64, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 53, in __init__ self.writer, self.log_txt = logger(self.exp_path, self.exp_name, self.pwd, 'exp', resume=cfg.RESUME) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py", line 77, in logger copy_cur_env(work_dir, exp_path+ '/' + exp_name + '/code', exception) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py", line 253, in copy_cur_env shutil.copytree(file, dst_file, dirs_exist_ok=True) TypeError: copytree() got an unexpected keyword argument 'dirs_exist_ok'The following error occurs, this is an error of python3.6.0, the error is resolved by upgrading to python3.7
The error message of this error does not show which position of the code I provided is wrong. The error caused seems to be an internal error in python
Was the release date of python 3.7.0 and was it before the code commit time?
Release date of python 3.7: June 27, 2018
Code submission time: Jul 2, 2019
Yes, python3.7 is released before the code submission time and can be installed
Exception in thread Thread-2: Traceback (most recent call last): File "/home/albert/anaconda3/envs/py360tc100/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/albert/anaconda3/envs/py360tc100/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/albert/anaconda3/envs/py360tc100/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG)) File "/home/albert/anaconda3/envs/py360tc100/lib/python3.6/signal.py", line 60, in pthread_sigmask sigs_set = _signal.pthread_sigmask(how, mask) ValueError: signal number 32 out of range Exception in thread Thread-2:Install a python3.7.0 virtual environment to solve the above error
conda create --name py370tc100 python=3.7.0Install torch1.0.0
This site https://pypi.org/project/torch/1.0.0/#files. Download torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl
pip install torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whlInstall other packages
tensorboardX
conda install tensorboardX==1.4easydict=1.9————
pip install easydict==1.9pandas ==0.24.2, Mar 14,2019
pip install pandas==0.24.2numpy == installed with pandas, the version is 1.19.2
scipy==1.2.1，Feb 9, 2019
pip install scipy==1.2.1run code
python train.pyThe error is as follows
Traceback (most recent call last): File "train.py", line 27, in <module> from datasets.SHHB.loading_data import loading_data File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/SHHB/loading_data.py", line 1, in <module> import torchvision.transforms as standard_transforms File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/__init__.py", line 2, in <module> from torchvision import datasets File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/datasets/__init__.py", line 9, in <module> from .fakedata import FakeData File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/datasets/fakedata.py", line 3, in <module> from .. import transforms File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/transforms/__init__.py", line 1, in <module> from .transforms import * File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 16, in <module> from . import functional as F File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/transforms/functional.py", line 5, in <module> from PIL import Image, ImageOps, ImageEnhance, PILLOW_VERSION ImportError: cannot import name 'PILLOW_VERSION' from 'PIL' (/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/PIL/__init__.py)Solution: In this file at this location /home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torchvision/transforms/functional.py", change the position PILLOW_VERSION in the figure to __version__
There are more thorough reforms in this URL https://blog.csdn.net/weixin_45021364/article/details/104600802; https://stackoverflow.com/questions/59659146/could-not-import-pillow-version -from-pil
Query the ubuntu version, and query the installation of the visualization tool occupied by the GPU capacity according to this version
Another error is reported, it is said that the GPU space is not enough?
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 66, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 94, in train pred_map = self.net(img, gt_map) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/CC.py", line 39, in forward density_map = self.CCN(img) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/SCC_Model/Res101_SFCN.py", line 46, in forward x = self.own_reslayer_3(x) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/SCC_Model/Res101_SFCN.py", line 124, in forward out = self.conv3(out) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 320, in forward self.padding, self.dilation, self.groups) RuntimeError: CUDA out of memory. Tried to allocate 48.00 MiB (GPU 0; 3.95 GiB total capacity; 3.44 GiB already allocated; 12.94 MiB free; 541.00 KiB cached)(Finally confirm) Read the code line by line again to make sure that the number of pictures read in a batch is at this position, and this file has been set as one
After the number of photos is changed to 60, the error of shutil is reported
Traceback (most recent call last): File "train.py", line 64, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 53, in __init__ self.writer, self.log_txt = logger(self.exp_path, self.exp_name, self.pwd, 'exp', resume=cfg.RESUME) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py", line 77, in logger copy_cur_env(work_dir, exp_path+ '/' + exp_name + '/code', exception) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py", line 251, in copy_cur_env shutil.copytree(file, dst_file) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/shutil.py", line 315, in copytree os.makedirs(dst) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/os.py", line 221, in makedirs mkdir(name, mode) FileExistsError: [Errno 17] File exists: './exp/12-14_15-25_SHHB_Res101_SFCN_1e-05/code/.git'two steps
—1—Add a parameter existing_ok=True to the place where shutil is called
Open this location, this file, /media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py,line251, change the above line to the following line . If there is a file in the target location, it will not be copied repeatedly
#shutil.copytree(file, dst_file) shutil.copytree(file, dst_file, dirs_exist_ok=True)—2—Take the hutil.copytree(file, dst_file) function of shutil in python 3.11.0
This location /home/albert/anaconda3/envs/py370tc100/lib/python3.7/shutil.py
Make a backup copy of the shutil.py file
The python official document has announced the code of shutil.py to be changed, to find out https://docs.python.org/3/library/shutil.html
The pasted things look like this
def _copytree(entries, src, dst, symlinks, ignore, copy_function, ignore_dangling_symlinks, dirs_exist_ok=False): if ignore is not None: ignored_names = ignore(os.fspath(src), [x.name for x in entries]) else: ignored_names = set() os.makedirs(dst, exist_ok=dirs_exist_ok) errors = [] use_srcentry = copy_function is copy2 or copy_function is copy for srcentry in entries: if srcentry.name in ignored_names: continue srcname = os.path.join(src, srcentry.name) dstname = os.path.join(dst, srcentry.name) srcobj = srcentry if use_srcentry else srcname try: is_symlink = srcentry.is_symlink() if is_symlink and os.name == 'nt': # Special check for directory junctions, which appear as # symlinks but we want to recurse. lstat = srcentry.stat(follow_symlinks=False) if lstat.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT: is_symlink = False if is_symlink: linkto = os.readlink(srcname) if symlinks: # We can't just leave it to `copy_function` because legacy # code with a custom `copy_function` may rely on copytree # doing the right thing. os.symlink(linkto, dstname) copystat(srcobj, dstname, follow_symlinks=not symlinks) else: # ignore dangling symlink if the flag is on if not os.path.exists(linkto) and ignore_dangling_symlinks: continue # otherwise let the copy occur. copy2 will raise an error if srcentry.is_dir(): copytree(srcobj, dstname, symlinks, ignore, copy_function, ignore_dangling_symlinks, dirs_exist_ok) else: copy_function(srcobj, dstname) elif srcentry.is_dir(): copytree(srcobj, dstname, symlinks, ignore, copy_function, ignore_dangling_symlinks, dirs_exist_ok) else: # Will raise a SpecialFileError for unsupported file types copy_function(srcobj, dstname) # catch the Error from the recursive copytree so that we can # continue with other files except Error as err: errors.extend(err.args[0]) except OSError as why: errors.append((srcname, dstname, str(why))) try: copystat(src, dst) except OSError as why: # Copying file access times may fail on Windows if getattr(why, 'winerror', None) is None: errors.append((src, dst, str(why))) if errors: raise Error(errors) return dst def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False, dirs_exist_ok=False): """Recursively copy a directory tree and return the destination directory. If exception(s) occur, an Error is raised with a list of reasons. If the optional symlinks flag is true, symbolic links in the source tree result in symbolic links in the destination tree; if it is false, the contents of the files pointed to by symbolic links are copied. If the file pointed by the symlink doesn't exist, an exception will be added in the list of errors raised in an Error exception at the end of the copy process. You can set the optional ignore_dangling_symlinks flag to true if you want to silence this exception. Notice that this has no effect on platforms that don't support os.symlink. The optional ignore argument is a callable. If given, it is called with the `src` parameter, which is the directory being visited by copytree(), and `names` which is the list of `src` contents, as returned by os.listdir(): callable(src, names) -> ignored_names Since copytree() is called recursively, the callable will be called once for each directory that is copied. It returns a list of names relative to the `src` directory that should not be copied. The optional copy_function argument is a callable that will be used to copy each file. It will be called with the source path and the destination path as arguments. By default, copy2() is used, but any function that supports the same signature (like copy()) can be used. If dirs_exist_ok is false (the default) and `dst` already exists, a `FileExistsError` is raised. If `dirs_exist_ok` is true, the copying operation will continue if it encounters existing directories, and files within the `dst` tree will be overwritten by corresponding files from the `src` tree. """ sys.audit("shutil.copytree", src, dst) with os.scandir(src) as itr: entries = list(itr) return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks, ignore=ignore, copy_function=copy_function, ignore_dangling_symlinks=ignore_dangling_symlinks, dirs_exist_ok=dirs_exist_ok)Then report an error saying 'sys' has no attribute 'audit', you go to shutil.py to comment out this sentence "sys.audit("shutil.copytree", src, dst)"
Traceback (most recent call last): File "train.py", line 64, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 53, in __init__ self.writer, self.log_txt = logger(self.exp_path, self.exp_name, self.pwd, 'exp', resume=cfg.RESUME) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py", line 77, in logger copy_cur_env(work_dir, exp_path+ '/' + exp_name + '/code', exception) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/misc/utils.py", line 253, in copy_cur_env shutil.copytree(file, dst_file, dirs_exist_ok=True) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/shutil.py", line 372, in copytree sys.audit("shutil.copytree", src, dst) AttributeError: module 'sys' has no attribute 'audit'Another error is that the GPU capacity is not enough
When there are 60 pictures, it is as follows
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 66, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 94, in train pred_map = self.net(img, gt_map) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/CC.py", line 39, in forward density_map = self.CCN(img) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/SCC_Model/Res101_SFCN.py", line 44, in forward x = self.frontend(x) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 320, in forward self.padding, self.dilation, self.groups) RuntimeError: CUDA out of memory. Tried to allocate 2.81 GiB (GPU 0; 3.95 GiB total capacity; 867.61 MiB already allocated; 2.60 GiB free; 788.00 KiB cached)1 picture is like this
RuntimeError: CUDA out of memory. Tried to allocate 48.00 MiB (GPU 0; 3.95 GiB total capacity; 3.44 GiB already allocated; 12.94 MiB free; 541.00 KiB cached)2 sheets
RuntimeError: CUDA out of memory. Tried to allocate 96.00 MiB (GPU 0; 3.95 GiB total capacity; 3.45 GiB already allocated; 2.94 MiB free; 685.00 KiB cached)This is the case when there are 6 pictures
RuntimeError: CUDA out of memory. Tried to allocate 288.00 MiB (GPU 0; 3.95 GiB total capacity; 3.24 GiB already allocated; 218.94 MiB free; 775.00 KiB cached)This is the case when there are 12 pictures
、RuntimeError: CUDA out of memory. Tried to allocate 576.00 MiB (GPU 0; 3.95 GiB total capacity; 2.96 GiB already allocated; 478.94 MiB free; 27.77 MiB cached)When there are 60 pictures, it is as follows
RuntimeError: CUDA out of memory. Tried to allocate 2.81 GiB (GPU 0; 3.95 GiB total capacity; 867.61 MiB already allocated; 2.60 GiB free; 788.00 KiB cached)This is the case when there are 400 pictures
RuntimeError: CUDA out of memory. Tried to allocate 3.52 GiB (GPU 0; 3.95 GiB total capacity; 147.61 MiB already allocated; 3.31 GiB free; 788.00 KiB cached)
relative
| Number of pictures |
memory allocated by plan |
free memory |
|
| 1 |
48MB |
12.94MB |
|
| 2 |
96MB |
2.94MB |
|
| 6 |
288MB |
218MB |
|
| 12 |
576MB |
478MB |
|
| 400 |
3.5G |
3.31G |
If there is a dataset
| data set |
img |
density |
| SHHB |
200kB |
2.2m |
| UCF50 |
48kB-300kB |
900kB-5.1,MB |
| UCF-QNRF |
70kB-290kB |
1.1MB-6MB |
| WE |
23kB-71kB- |
800kB-1.3MB |
Try another smaller dataset
Try it with WE
This location /media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework, this file config.py, line13, changed to WE
#__C.DATASET = 'SHHB' __C.DATASET = 'WE'Run it and get the following error
Traceback (most recent call last): File "train.py", line 64, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 40, in __init__ self.train_loader, self.val_loader, self.restore_transform = dataloader() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/WE/loading_data.py", line 30, in loading_data train_set = WE(cfg_data.DATA_PATH+'/train', 'train',main_transform=train_main_transform, img_transform=img_transform, gt_transform=gt_transform) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/WE/WE.py", line 18, in __init__ self.data_files = [filename for filename in os.listdir(self.img_path) \ FileNotFoundError: [Errno 2] No such file or directory: '/media/D/DataSet/CC/WE_2019/train/img'It seems that I can't find where the data set exists
We go to
/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/WEline10 of "setting.py" at this location ""
# __C_WE.DATA_PATH = '/media/D/DataSet/CC/WE_2019' __C_WE.DATA_PATH = '../ProcessedData/WE_blurred'Report the error of insufficient GPU memory again
RuntimeError: CUDA out of memory. Tried to allocate 126.00 MiB (GPU 0; 3.95 GiB total capacity; 3.34 GiB already allocated; 100.94 MiB free; 773.00 KiB cached)Reduce the number of pictures per load of WE to 1
" " line21 at file location "
/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/WE//"setting.py
#__C_WE.TRAIN_BATCH_SIZE = 12 #imgs __C_WE.TRAIN_BATCH_SIZE = 1 #__C_WE.VAL_BATCH_SIZE = 8 # __C_WE.VAL_BATCH_SIZE = 1In order not to let the code run for too long, I set epoch to 1
location"
model_innov/2021_C-3-Framework/C-3-Framework”config.pyline43
#__C.MAX_EPOCH = 200 __C.MAX_EPOCH = 1Run it again, it works - it works
The name of the pre-trained model of Res101_SFCN resnet101-5d3b4d8f.pth
The location where the trained model is stored:
/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/exp/12-14_21-22_WE_Res101_SFCN_1e-05/all_ep_1_mae_19.3_mse_0.0.pthIf the video memory on colab is greater than 4GB, you can use colab
Change the model MCNN to train with the WE data set
Location:
/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/config.pyLine26
# __C.NET = 'Res101_SFCN' # net selection: MCNN, AlexNet, VGG, VGG_DECODER, Res50, CSRNet, SANet __C.NET = MCNNoperation result
Use the WE dataset to train on the AlexNet model
Download the pre-trained model from this URL https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth and put the file in this location
/home/albert/.torch/modelsWE dataset training VGG model
Pre-trained model file https://download.pytorch.org/models/vgg16-397923af.pth
Train the VGG_DECODER model on the WE dataset
Train the Res50 model on the WE dataset
Pre-training model code: https://download.pytorch.org/models/resnet50-19c8e357.pth
Run CSRNet on the WE dataset
Run SANet on WE dataset
report error
Traceback (most recent call last): File "train.py", line 64, in <module> cc_trainer = Trainer(loading_data,cfg_data,pwd) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer_for_M2TCC.py", line 31, in __init__ self.net = CrowdCounter(cfg.GPU_ID,self.net_name,loss_1_fn,loss_2_fn).cuda() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/M2TCC.py", line 13, in __init__ from M2TCC_Model.SANet import SANet as net ModuleNotFoundError: No module named 'M2TCC_Model'In the line13 of M2TCC.py in the models folder, add a model before the import module.. because this code is in this location /media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/ C-3-Framework/train.py, here you need to enter the models folder first, then you can enter the M2TCC_Model folder, and finally enter SANet.py, so add models in front.
if model_name == 'SANet': # newly modified 2022-12-17-8：44 #from M2TCC_Model.SANet import SANet as net from models.M2TCC_Model.SANet import SANet as netLine 193 of trainer_for_M2TCC.py, add a [0], because self.net.loss returns a tuple with only one element, add a [0] to take out the elements in the tuple, and then use item take things out
#losses.update(self.net.loss.item(),i_sub) losses.update(self.net.loss[0].item(),i_sub)Then it works successfully
available models
| Word Expo dataset |
||||||
| Model |
a batch |
total train time |
Final saved model file size |
Pretrained model size |
GPU usage percentage |
MAE |
| Res101_SFCN |
0.95s |
3147s |
155MB |
180MB |
19.33 |
|
| MCNN |
0.13s |
364s |
540kB |
none |
35.54 |
|
| AlexNet |
0.03s |
12s |
10MB |
245MB |
19.88 |
|
| VGG |
0.36s |
760s |
30MB |
553MB |
1879M.100% |
14.17 |
| VGG_DECODER |
0.4s |
850s |
34MB |
I don't know if I use VGG |
1989M 100% |
64.78 |
| Res50 |
0.24s |
800s |
100MB |
100MB |
1625M 99% |
20.14 |
| CSRNet |
0.73s |
1500s |
65MB |
Might be, I don't know which one. |
2073MB 100% |
16.07 |
| SANet |
0.4s |
970s |
5.6MB |
Might be, I don't know which one. |
1633MB 100% |
38.3 |
——Check the training curve with TensorBoardX
Because tensorboard is not installed, an error is reported, and you are required to install it
What version of tensorboardX are you installing now?
——1.4 (release time: (2018-08-09))
——How to check the installed version of tensorboardX? ——To tell you the truth, I didn’t find a way to query the version number——Which version number of the tensorboard should be installed corresponding to this tensorboardX? ——To install tensorboard, tensorflow must be installed. It is better to install tensorflow directly and install tensorboard by the way.
Command to query tensorboard version: from tensorboard import version; print(version.VERSION)
The tensorflow package started to support python3.7 from 1.1.3.1 (2019-02-27). The previous version does not support it, and you cannot install it
pip install tensorflow==1.13.1It can be run, but the picture made is useless - I only ran one epoch, and I definitely can't see it. If there are more epochs, I can see it naturally
——You take a few models with the shortest time to run a batch, try to increase the number of pictures in a batch to 3, 4, until the maximum number of pictures that can be run, I want to see if the 4GB video memory can be filled. ——Improve training efficiency
change this position
/2021_C-3-Framework/C-3-Framework/datasets/WE/setting.py
__C_WE.RESUME_MODEL = ''#model path #__C_WE.TRAIN_BATCH_SIZE = 6 #imgs __C_WE.TRAIN_BATCH_SIZE = 6 #__C_WE.VAL_BATCH_SIZE = 6 # __C_WE.VAL_BATCH_SIZE = 6What model to use for this position
2021_C-3-Framework/C-3-Framework/config.pyUsing AlexNet, it has the smallest size and the fastest batch processing
__C.NET = 'AlexNet'
| batchsize |
Video memory occupies 4GB |
||
| AlexNet |
6 |
947MB/4GB |
|
| AlexNet |
24 |
2151MB |
|
| AlexNet |
48 |
3297MB |
|
| 55 |
3555MB |
||
| 60 |
3923MB |
||
| 70 |
3515MB |
||
| 80 |
4025MB |
||
| 85 |
No error, the computer is stuck and cannot run |
This illustrates one point. It is wrong to say that the GPU usage is 100%. It shows how much MB is actually occupied. Only when 4 GB is filled is the maximum utilization.
——For the remaining data sets, try to run with a data set with a relatively small model to see if it can run? ——Run out, indicating that a data set can be used (small models can be run directly, indicating that there is no problem with my code, it’s just that the video memory is not enough; large models can be rented to run on the server)
Under the AlexNet model, batch_size=1
Change the data set, in
2021_C-3-Framework/C-3-Framework/config.py
__C.DATASET = 'WE'
SHHA error
I also don't know how this mat file should be
Traceback (most recent call last): File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in <listcomp> samples = collate_fn([dataset[i] for i in batch_indices]) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/SHHA/SHHA.py", line 27, in __getitem__ img, den = self.read_image_and_gt(fname) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/datasets/SHHA/SHHA.py", line 44, in read_image_and_gt den = sio.loadmat(os.path.join(self.gt_path,os.path.splitext(fname)[0] + '.mat')) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/scipy/io/matlab/mio.py", line 207, in loadmat MR, file_opened = mat_reader_factory(file_name, appendmat, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/scipy/io/matlab/mio.py", line 62, in mat_reader_factory byte_stream, file_opened = _open_file(file_name, appendmat) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/scipy/io/matlab/mio.py", line 37, in _open_file return open(file_like, 'rb'), True FileNotFoundError: [Errno 2] No such file or directory: '../ProcessedData/shanghaitech_part_A/train/den/254.mat'
Found the reason: line47 of the SHHA.py file, comment the first two lines, open the last one and read the csv file
#den = sio.loadmat(os.path.join(self.gt_path,os.path.splitext(fname)[0] + '.mat')) #den = den['map'] den = pd.read_csv(os.path.join(self.gt_path,os.path.splitext(fname)[0] + '.csv'), sep=',',header=None).valuesAnother error is reported that the dimensions are inconsistent
How can it be inconsistent after calculation? I don't understand
I suspect that the ground truth and the predicted picture he took are not the same picture, so the size does not match
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 66, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/trainer.py", line 94, in train pred_map = self.net(img, gt_map) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/CC.py", line 40, in forward self.loss_mse= self.build_loss(density_map.squeeze(), gt_map.squeeze()) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2021_C-3-Framework/C-3-Framework/models/CC.py", line 44, in build_loss loss_mse = self.loss_mse_fn(density_map, gt_data) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 435, in forward return F.mse_loss(input, target, reduction=self.reduction) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/functional.py", line 2155, in mse_loss expanded_input, expanded_target = torch.broadcast_tensors(input, target) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/functional.py", line 50, in broadcast_tensors return torch._C._VariableFunctions.broadcast_tensors(tensors) RuntimeError: The size of tensor a (688) must match the size of tensor b (683) at non-singleton dimension 0
UCF50
| AlexNet+ dataset |
|
| WE |
OK |
| SHHB |
OK |
| SHHA |
Report an error, I didn't fix it |
| UCF50 |
OK |
| QNRF |
OK |
| Mall |
unprocessed data |
| UCSD |
unprocessed data |
——For these eight models, take a look at the theoretical description for a while, and look at the code for a while. (1) I understand the code and can reuse it in the future; (2) I understand the code of other people’s models. In order to improve on the basis of other people's models
Read the code line by line to achieve a one-to-one correspondence between the model theory and the code
Under the WE dataset, try the three models,
The existing codes implement these models for crowd counting: MCNN, CSRNet, 'Res101_SFCN,, AlexNet, VGG, VGG_DECODER, Res50, SANet
I would like to know how your MCNN code is implemented?
train.py ——> The following models all go through trainer.py-'MCNN', 'AlexNet', 'VGG', 'VGG_DECODER', 'Res50', 'Res101', 'CSRNet','Res101_SFCN' ——> It is MCNN ——> trainer.py ——> Are the codes of all these models the same?
(Universal Code) 2019_Video-Crowd-Counting
目前这个代码被存放在我电脑的这个位置：/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting
Since current datasets are basic based on single image, there are only Mall, UCSD, FDST are video based
FDST
配环境
要求的环境是这样，但是建议用python3.7先跑一下，不行再换3.6。要求安装的包大概是tensorboard，easydict,这一类的——不用换用python3.7那个即可
conda create -n cascadecnn python=3.6 pip install -r requirements.txt
下载数据集
视频人群计数用的数据集是 Mall and FDST.
用代码预处理数据集
他这个代码不完整，他提到的这两个文件夹，他并没有提供
我自己新建文件夹把原有文件夹里这些文件移动进去，自己去创个文件夹
你去仿照出C3F的代码，人家datasets里面别人有什么你就用组建什么
config.py的line28标注了预训练模型应该存储在这个位置'./best_pre_model/Mall_pre_model.pth'‘
把FDST数据集下载下来址（https://github.com/sweetyy83/Lstn_fdst_dataset）
用MATLAB软件的这个代码PrepareFDST.m把FDST数据集处理一遍
MATLAB添加一个包jsonlab-1.5，因为代码里有用到这个包
链接：https://pan.baidu.com/s/1dZBi5j04dMLW3huxY8uhvQ 提取码：o909 把jsonlab-1.5放在这个位置F:\Programming_software\MATLAB\toolbox 在MATLAB命令行执行 addpath('F:\Programming_software\MATLAB\toolbox\jsonlab-1.5') savepath 刷新缓存 rehash toolboxcache 然后在MATLAB代码的line3更改路径如下 addpath('F:\Programming_software\MATLAB\toolbox\jsonlab-1.5')
在同一文件夹下建立下列文件夹
\FDST\train\den；\FDST\test\den
在PrepareFDST.m里面的line8，train改成test就能测试集也制作一遍
你要把用MATLAB处理过FDST数据集，放到这个位置datasets/ProcessedData/FDST/train/
然后运行
python train.py
报错
Traceback (most recent call last): File "train.py", line 64, in <module> cc_trainer = Trainer(loading_data('DA'),cfg_data,pwd) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 25, in __init__ self.net = CrowdCounter(cfg.GPU_ID,self.net_name).cuda() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/CC.py", line 31, in __init__ self.CCN = torch.nn.DataParallel(self.CCN, device_ids=gpus).cuda() File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 131, in __init__ _check_balance(self.device_ids) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 18, in _check_balance dev_props = [torch.cuda.get_device_properties(i) for i in device_ids] File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 18, in <listcomp> dev_props = [torch.cuda.get_device_properties(i) for i in device_ids] File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/cuda/__init__.py", line 301, in get_device_properties raise AssertionError("Invalid device id") AssertionError: Invalid device id
去config.py的line37
#__C.GPU_ID = [0,1,2,3] # sigle gpu: [0], [1] ...; multi gpus: [0,1] __C.GPU_ID = [0]
再次运行，再次报错
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 69, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 102, in train pred_map = self.net(img, gt_map, img_p) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/CC.py", line 41, in forward density_map = self.CCN(img,img_p) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py", line 124, in forward x_p = self.stn(x_p) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py", line 99, in stn xs = xs.view(-1, 10*11*16) # for Mall, it is 10*11*16, for FDST it's 10*7*16 RuntimeError: shape '[-1, 1760]' is invalid for input of size 8960
我的感觉是数据的维度不匹配所以报错
这个位置这个文件/models/SCC_Model/CascadeCNN.py 的line99,改成作者介绍的维度
# line99 #xs = xs.view(-1, 10*11*16) # for Mall, it is 10*11*16, for FDST it's 10*7*16 xs = xs.view(-1, 10*7*16) # line114 #xs = xs.view(-1, 10*11*16) xs = xs.view(-1, 10*7*16)
依旧报错，还是维度不匹配
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 69, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 102, in train pred_map = self.net(img, gt_map, img_p) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/CC.py", line 41, in forward density_map = self.CCN(img,img_p) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py", line 133, in forward x_p = self.stn(x_p) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py", line 104, in stn theta = self.fc_loc(xs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 67, in forward return F.linear(input, self.weight, self.bias) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/functional.py", line 1352, in linear ret = torch.addmm(torch.jit._unwrap_optional(bias), input, weight.t()) RuntimeError: size mismatch, m1: [1 x 1120], m2: [1760 x 32] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:266
在/models/SCC_Model/CascadeCNN.py的line26,维度改成这样
self.fc_loc = nn.Sequential( #nn.Linear(10 * 11 * 16, 32), nn.Linear(10 * 7 * 16, 32),
继续报错
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 69, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 102, in train pred_map = self.net(img, gt_map, img_p) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/CC.py", line 41, in forward density_map = self.CCN(img,img_p) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py", line 138, in forward x_p1 = self.stn_early(x_p1) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py", line 124, in stn_early theta = self.fc_loc_1(xs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 67, in forward return F.linear(input, self.weight, self.bias) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/nn/functional.py", line 1352, in linear ret = torch.addmm(torch.jit._unwrap_optional(bias), input, weight.t()) RuntimeError: size mismatch, m1: [1 x 1120], m2: [1760 x 32] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:266
在/models/SCC_Model/CascadeCNN.py的line46,维度改成这样。这样的话输入的维度和神经网络的维度就匹配了
self.fc_loc_1 = nn.Sequential( # nn.Linear(10 * 11 * 16, 32), nn.Linear(10 * 7 * 16, 32),
然后就可以跑通了
做一些轻微的优化
max_epoch改成1
config.py line45 #__C.MAX_EPOCH = 200 __C.MAX_EPOCH = 1
batch_size改成4
#__C_MALL.TRAIN_BATCH_SIZE = 1 #imgs __C_MALL.TRAIN_BATCH_SIZE = 4 #__C_MALL.VAL_BATCH_SIZE = 1 # __C_MALL.VAL_BATCH_SIZE = 4
pretrain_model把VGG下载下来，并允许使用VGG网络的预训练模型参数
/models/SCC_Model/CascadeCNN.py line9 # model_path = '../PyTorch_Pretrained/vgg16-397923af.pth' model_path = './home/albert/.torch/models/vgg16-397923af.pth'
运行，报错
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 69, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 93, in train for i, data in enumerate(self.train_loader, 0): File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 615, in __next__ batch = self.collate_fn([self.dataset[i] for i in indices]) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 615, in <listcomp> batch = self.collate_fn([self.dataset[i] for i in indices]) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/datasets/FDST/FDST.py", line 37, in __getitem__ pname = self.data_files[index] IndexError: list index out of range
——打印出来，看看index是什么样,我是怎么超的，不超的话数字应该是哪个？，怎么改？——限制一个epoch只跑10个数据——什么地方跑的那个循环，什么地方决定了先这张图再那张图？
循环的地方位于“trainer.py”的line97
在trianer.py的line99加入下面这行代码，让数据集在吃了20个数据以后停止训练
if i == 20:break
然后运行报了这个错
Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 87, in forward torch.save(self.net.cpu().state_dict(),"./weights/Pre_model_{}.pth".format(epoch+1)) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/serialization.py", line 218, in save return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol)) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/serialization.py", line 141, in _with_file_like f = open(f, mode) FileNotFoundError: [Errno 2] No such file or directory: './weights/Pre_model_1.pth'
应该是这个文件夹不存在/weights/，模型保存不下来。在train.py同一路径下新建这个文件夹weights。再次运行
正常运行了，没报错
把train的数据换成200条数据
trainer.py, line101 if i == 200:break
可以正常运行
去掉读取数据的限制，整个数据集读取
又开始报错，在跑数据，跑到4500的时候报这个错
[ep 1][it 4460][loss 0.0030][lr 0.1000][0.20s] [cnt: gt: 27.0 pred: 24.70] [ep 1][it 4470][loss 0.0041][lr 0.1000][0.20s] [cnt: gt: 36.0 pred: 36.27] [ep 1][it 4480][loss 0.0049][lr 0.1000][0.21s] [cnt: gt: 42.9 pred: 40.18] [ep 1][it 4490][loss 0.0032][lr 0.1000][0.15s] [ep 1][it 4500][loss 0.0052][lr 0.1000][0.12s] [cnt: gt: 41.0 pred: 37.46] Traceback (most recent call last): File "train.py", line 65, in <module> cc_trainer.forward() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 70, in forward self.train() File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/trainer.py", line 97, in train for i, data in enumerate(self.train_loader, 0): File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 615, in __next__ batch = self.collate_fn([self.dataset[i] for i in indices]) File "/home/albert/anaconda3/envs/py370tc100/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 615, in <listcomp> batch = self.collate_fn([self.dataset[i] for i in indices]) File "/media/F:/FILES_OF_ALBERT/IT_paid_class/graduation_thesis/model_innov/2019_Video-Crowd-Counting/Video-Crowd-Counting/datasets/FDST/FDST.py", line 39, in __getitem__ pname = self.data_files[index] IndexError: list index out of range
再跑一次，我看看这次停在多少条个数据集那里
这个位置Video-Crowd-Counting/datasets/FDST/FDST.py 的line39我把index打印出来了，发现index是一个介于0-5000的随机数字
我目前的怀疑是FDST制作ground truth的时候，没有使用整个数据集，而是从里面有选择性的挑选了一部分来作为数据集，这样的话就会出现，你给的index比数据集中数据的数量多，list out of range的错误————去MATLAB代码那里做调整使得制作ground truth的时候拿数据集的全量数据集来制作——存在这个问题，但是 list index out of range不是这个问题，，是index从1开始导致的
原始数据集
train：1，2，3，6，7，8，11，12，13，16，17，18，21，22，23，26，27，28，，31，32，33，36，37，38，41，42，43，46，47，48，51，52，53，56，57，，58，61，62
加工过的数据集
train：6（1-150），7（1-150），8，47，48，51，52，53，56，57，58，61，62，
——他确实是在加工原始数据集的过程中丢弃了原有的一部分数据集，像上面红色编号的这些。但是我看了他的MATLAB代码，没有找到哪行代码是用来选择使用哪部分数据集、丢弃哪部分数据集的。我也很疑惑，他是怎么做到的。
报错前最后一个indecent是5250有没有可能是因为，index是从0开始的，最后一个index应该是5249？——把index减去1，是这样，按照这样改了以后就不报错了
/Video-Crowd-Counting/datasets/FDST/FDST.py的line46 给index-1
# pname = self.data_files[index] pname = self.data_files[index-1] # modified by Albert # fname = self.data_files[index-1] # 为啥要减1我不知道 fname = self.data_files[index-2] # 我的问题是，为什么pname是data_files[index-1]，为甚fname是fname = self.data_files[index-2]？
确实不报错了，但是模型似乎没有任何用处了，损失值很高，就跟模型没训练过一样
print(index) 3102 print(index) train time: 1747.25s ==================== Best model: 0 MAE: 100000.0 MSE: 100000.0
我尝试换成这样改，还是把index减一，让它从0开始；前面那个减一给他恢复
# 原来没有注释，现在注释掉。不然最后一张图片会超index # if num == 0: # index += 1 pname = self.data_files[index] # modified by Albert # pname = self.data_files[index-1] # modified by Albert fname = self.data_files[index-1] # 为啥要减1我不知道 # fname = self.data_files[index-2]
看看结果如何？依旧是和没建模一样，损失值高的离谱
[cnt: gt: 39.8 pred: 36.79] [ep 1][it 1270][loss 0.0037][lr 0.1000][0.72s] [cnt: gt: 30.0 pred: 28.50] [ep 1][it 1280][loss 0.0036][lr 0.1000][0.50s] [cnt: gt: 26.0 pred: 22.78] [ep 1][it 1290][loss 0.0045][lr 0.1000][0.77s] [cnt: gt: 37.9 pred: 35.58] [ep 1][it 1300][loss 0.0037][lr 0.1000][0.85s] [cnt: gt: 18.0 pred: 18.69] [ep 1][it 1310][loss 0.0038][lr 0.1000][0.79s] [cnt: gt: 22.0 pred: 22.20] train time: 1780.58s ==================== Best model: 0 MAE: 100000.0 MSE: 100000.0 训练一个epoch是30分钟
确认一下FDST的train和test和你制作后的变换是否一一匹配？
使用try ... continue，如果遇到 list out of range的错误，自动index减一，避免这个错误引发的报错
try: pname = self.data_files[index] fname = self.data_files[index-1] # 为啥要减1我不知道 # fname = self.data_files[index-2] # modified by Albert except IndexError: print("IndexError is raised and the index number is ",index) pname = self.data_files[index-1] fname = self.data_files[index - 2] # 为啥要减1我不知道
再次运行，还是损失值很高。不知道如何处理
[ep 1][it 1240][loss 0.0049][lr 0.1000][0.86s] [cnt: gt: 29.0 pred: 24.67] [ep 1][it 1250][loss 0.0036][lr 0.1000][0.87s] [cnt: gt: 30.0 pred: 26.48] [ep 1][it 1260][loss 0.0046][lr 0.1000][0.90s] [cnt: gt: 39.9 pred: 36.57] [ep 1][it 1270][loss 0.0038][lr 0.1000][0.87s] [cnt: gt: 30.0 pred: 30.45] [ep 1][it 1280][loss 0.0036][lr 0.1000][0.86s] [cnt: gt: 26.0 pred: 24.42] [ep 1][it 1290][loss 0.0043][lr 0.1000][0.90s] [cnt: gt: 37.7 pred: 32.65] [ep 1][it 1300][loss 0.0036][lr 0.1000][0.84s] [cnt: gt: 18.0 pred: 17.51] [ep 1][it 1310][loss 0.0038][lr 0.1000][0.86s] [cnt: gt: 22.0 pred: 20.35] train time: 1740.76s ==================== Best model: 0 MAE: 100000.0 MSE: 100000.0 我现在很怀疑 gt:22是指的实际的人数，pred指的是预测的人数
——有没有可能是训练的epoch太少，还没收敛？有可能，官方的训练epoch数是30
训练个30epoch，15小时，看看最后的损失值
config.py line30
#__C.MAX_EPOCH = 1 __C.MAX_EPOCH = 30
训练结束MAE MSE真的降低下来了
[ep 30][it 1250][loss 0.0036][lr 0.0865][0.91s] [cnt: gt: 29.0 pred: 31.48] [ep 30][it 1260][loss 0.0040][lr 0.0865][0.92s] [cnt: gt: 36.0 pred: 33.92] [ep 30][it 1270][loss 0.0038][lr 0.0865][0.90s] [cnt: gt: 32.0 pred: 27.08] [ep 30][it 1280][loss 0.0024][lr 0.0865][0.94s] [cnt: gt: 24.0 pred: 28.90] [ep 30][it 1290][loss 0.0034][lr 0.0865][0.91s] [cnt: gt: 53.0 pred: 49.13] [ep 30][it 1300][loss 0.0034][lr 0.0865][0.91s] [cnt: gt: 39.0 pred: 38.39] [ep 30][it 1310][loss 0.0037][lr 0.0865][0.92s] [cnt: gt: 23.0 pred: 22.75] train time: 1765.95s ==================== Best model: 10 MAE: 1.7196840407986116 MSE: 2.1560562334881412
他这个模型是“真的”video crowd counting吗？丢进去一短视频的录像能用这个模型数出人数吗
目前的问题是，模型损失值居高不下，不收敛
——逐行查看，看看有没有被注释，可以用的其他代码
——VGG预训练模型加载的那句话注释掉，训练 一个epoch看看分数——我这次没有注释，分数降下了；是不是注释这句话，第一个epoch损失值就降下来了呢？（因为我怀疑vgg-16的预训练参数并不适用于我这个人群计数的场景）
这个位置这个文件/Video-Crowd-Counting/models/SCC_Model/CascadeCNN.py line11注释掉
# model_path = '../PyTorch_Pretrained/vgg16-397923af.pth' # model_path = './home/albert/.torch/models/vgg16-397923af.pth'
config.py的line49改30为1
__C.MAX_EPOCH = 1
刚开始训练的情况
[ep 1][it 10][loss 0.0191][lr 0.1000][0.45s] [cnt: gt: 38.0 pred: 149.47] [ep 1][it 20][loss 0.0130][lr 0.1000][0.76s] [cnt: gt: 27.0 pred: 56.28] [ep 1][it 30][loss 0.0102][lr 0.1000][0.82s] [cnt: gt: 35.0 pred: 30.14] [ep 1][it 40][loss 0.0088][lr 0.1000][0.81s] [cnt: gt: 29.0 pred: 65.56] [ep 1][it 50][loss 0.0082][lr 0.1000][0.80s] [cnt: gt: 35.0 pred: 20.24] [ep 1][it 60][loss 0.0079][lr 0.1000][0.87s] [cnt: gt: 22.0 pred: 12.62] [ep 1][it 70][loss 0.0069][lr 0.1000][0.71s] [cnt: gt: 31.0 pred: 20.80] [ep 1][it 80][loss 0.0075][lr 0.1000][0.73s] [cnt: gt: 27.0 pred: 2.85] [ep 1][it 90][loss 0.0068][lr 0.1000][0.72s] [cnt: gt: 30.0 pred: 6.35] [ep 1][it 100][loss 0.0055][lr 0.1000][0.45s] [cnt: gt: 11.0 pred: 10.80] [ep 1][it 110][loss 0.0054][lr 0.1000][0.61s] [cnt: gt: 38.0 pred: 39.72] [ep 1][it 120][loss 0.0069][lr 0.1000][0.59s] [cnt: gt: 46.0 pred: 41.28] [ep 1][it 130][loss 0.0078][lr 0.1000][0.65s] [cnt: gt: 36.0 pred: 23.80] [ep 1][it 140][loss 0.0056][lr 0.1000][0.77s] [cnt: gt: 23.0 pred: 14.88] [ep 1][it 150][loss 0.0064][lr 0.1000][0.46s] [cnt: gt: 32.0 pred: 23.16] [ep 1][it 160][loss 0.0062][lr 0.1000][0.60s] [cnt: gt: 25.0 pred: 15.43] [ep 1][it 170][loss 0.0055][lr 0.1000][0.61s] [cnt: gt: 31.0 pred: 21.56] [ep 1][it 180][loss 0.0073][lr 0.1000][0.57s] [cnt: gt: 38.0 pred: 22.98]
一个epoch训练出来损失值依然很高。说明不运用预训练模型参数对于加速训练是没有用的，一个epoch依然不收敛
[ep 1][it 1260][loss 0.0048][lr 0.1000][0.89s] [cnt: gt: 39.9 pred: 31.28] [ep 1][it 1270][loss 0.0039][lr 0.1000][0.78s] [cnt: gt: 30.0 pred: 27.76] [ep 1][it 1280][loss 0.0036][lr 0.1000][0.77s] [cnt: gt: 26.0 pred: 24.10] [ep 1][it 1290][loss 0.0043][lr 0.1000][0.86s] [cnt: gt: 37.7 pred: 37.51] [ep 1][it 1300][loss 0.0037][lr 0.1000][0.83s] [cnt: gt: 18.0 pred: 18.61] [ep 1][it 1310][loss 0.0037][lr 0.1000][0.87s] [cnt: gt: 22.0 pred: 19.71] train time: 1806.04s ==================== Best model: 0 MAE: 100000.0 MSE: 100000.0
——有没有可能是作者损失函数在最后打印的那个定义错了？
——用test.py测一下，拿着训练出的这些模型，看看每个模型多少分数？
——如果FDST不能跑，那么换成Mall数据集跑出来什么样，文件路径、尺寸等等都修改了，是否还训练结束了损失分数特别高？
——我怀疑目前训练的数据集和ground truth之间的匹配关系是错的，（比如A图片和B图片的标注来对应），导致怎么训练损失值都是很高的。——逐行读代码，看看能不能找到什么地方做的匹配。——通过目前的分数看不是这样，但是具体是怎匹配的，我还不知道
/Video-Crowd-Counting/datasets/FDST 的line48和line52为什么要 index 和index-1来表示picture和frame？ 是不是二者完全是驴唇不对马嘴的不相干的数据
——逐行读懂single image crowd counting的代码
——回来再尝试解决 video crowd counting的代码的问题
——这个video CC的代码解决不了，那么去找其他的video crowd counting的代码，尝试无报错跑通
Mall
用MATLAB处理一下Ma什么地方ll的原始数据集
修改读取数据集的文件夹 文件目录位置
修改数据的维度
运行
这里
多查几篇论文看看他们是否提供代码，把提供代码的这些模型逐一列出来，逐一跑。
运行代码
(万能代码)gjy3035/NWPU-Crowd-Sample-Code
gjy3035/NWPU-Crowd-Sample-Code
数据集：https://gjy3035.github.io/NWPU-Crowd-Sample-Code/
(万能代码)surajdakua/Crowd-Counting-Using-Pytorch
https://github.com/surajdakua/Crowd-Counting-Using-Pytorch
https://www.analyticsvidhya.com/blog/2019/02/building-crowd-counting-model-python/
(万能代码)miao0913/C3-framework-trees
对于数据预处理、训练测试全部都有jupyter notebook做讲解，很好用
CUDA要求的版本低，得装双cuda
运行代码的指导https://medium.com/@kaamyakpant_67666/building-a-crowd-counting-model-using-deep-learning-e8b8e925674e
（1）Creating density map—— follow the make_dataset.ipynb to generate the ground truth. It shall take some time to generate the dynamic ground truth
Python: 2.7 PyTorch: 0.4.0 CUDA: 9.2
(万能代码)MRJTM/crowd_counting
https://github.com/MRJTM/crowd_counting
(万能代码)2020_gaoguangshuai/survey-for-crowd-counting
2020_cite=125_未发表_Gao——Cnn-based density estimation and crowd counting: A survey
我们在NWPU数据集的验证集中提供了一些主流算法,（1）如何生成密度图的代码和如何生成人数预测结果的代码，（3）还提供了评估预测好坏的工具
gaoguangshuai/survey-for-crowd-counting
百度Paddle下FairMOT模型
先把百度这个现成的代码给我运行起来：https://aistudio.baidu.com/aistudio/projectdetail/2421822
tensorFlow和pytorch官方有没有人群计数crowd counting的官方示例——谷歌英文博客有没有博客既有解释、又有代码的，个人用pytorch和tensorflow实现的
-
代码怎么跑说的很详细（jupyer notebook）：https://aistudio.baidu.com/aistudio/projectdetail/2421822
-
写的比下面这个都更加详细：https://aistudio.baidu.com/aistudio/projectdetail/4171185
-
PaddlePaddle/PaddleDetectionhttps://github.com/PaddlePaddle/paddledetection#%E4%BA%
-
PP-Human实时行人分析全流程实战：写的特别详细，怎么安装、怎么运行写的都很好：https://aistudio.baidu.com/aistudio/projectdetail/3842982 ——;
-
实时行人分析 PP-Human——https://toscode.gitee.com/zyt111/PaddleDetection/tree/release/2.4/deploy/pphuman
-
使用百度AI实现视频的人流量统计（静态+动态）代码及效果演示 https://blog.csdn.net/weixin_419
-
基于PaddleDetection实现人流量统计人体检测https://blog.csdn.net/m0_63642362/article/details/121434604
（文章写的很详细，各部分都很完善，非常好！）
人流量统计任务需要在
（1）检测到目标的类别和位置信息的同时
（2）识别出帧与帧间的关联信息，确保视频中的同一个人不会被多次识别并计数。本案例选取PaddleDetection目标跟踪算法中的FairMOT模型来解决人流量统计问题。
（3）FairMOT以Anchor Free的CenterNet检测器为基础，
（4）深浅层特征融合使得检测和ReID任务各自获得所需要的特征，实现了两个任务之间的公平性，并获得了更高水平的实时多目标跟踪精度。
（5）针对拍摄角度不同（平角或俯角）以及人员疏密程度，在本案例设计了不同的训练方法：
-
针对人员相对稀疏的场景： 基于Caltech Pedestrian、CityPersons、CHUK-SYSU、PRW、ETHZ、MOT16和MOT17数据集进行训练，对场景中的行人进行全身检测和跟踪。 如 图2 所示，模型会对场景中检测到的行人进行标识，并在左上角显示出该帧场景下的行人数量，实现人流量统计。
-
针对人员相对密集的场景： 人与人之间的遮挡问题会非常严重，这时如果选择对行人整体检测，会导致漏检率升高。因此，本场景中使用人头跟踪方法。基于HT-21数据集进行训练，对场景中的行人进行人头检测和跟踪，对人流量的统计基于检测到的人头进行计数，如 图3 所示。
模型选择
PaddleDetection对于多目标追踪算法主要提供了三种模型，DeepSORT、JDE和FairMOT。
-
DeepSORT (Deep Cosine Metric Learning SORT) （1）扩展了原有的 SORT (Simple Online and Realtime Tracking) 算法，（2）增加了一个CNN模型用于在检测器限定的人体部分图像中提取特征，在深度外观描述的基础上整合外观信息，将检出的目标分配和更新到已有的对应轨迹上即进行一个ReID重识别任务。DeepSORT所需的检测框可以由任意一个检测器来生成，然后读入保存的检测结果和视频图片即可进行跟踪预测。ReID模型此处选择 PaddleClas 提供的
PCB+Pyramid ResNet101模型。 -
JDE (Joint Detection and Embedding) 是（1）在一个单一的共享神经网络中同时学习目标检测任务（Anchor Base的YOLOv3检测器）和embedding任务（ReID分支学习embedding），（2）并同时输出检测结果和对应的外观embedding匹配的算法。一个模型有两个输出，所以训练过程被构建为一个多任务联合学习问题。这样做的好处：兼顾精度和速度。
-
FairMOT 【最后用这个】（1）以Anchor Free的CenterNet检测器为基础，（2）克服了Anchor-Based的检测框架中anchor和特征不对齐问题，（3）深浅层特征融合使得检测和ReID任务各自获得所需要的特征，（4）并且使用低维度ReID特征，（5）提出了一种由两个同质分支组成的简单baseline来预测像素级目标得分和ReID特征，实现了两个任务之间的公平性，并获得了更高水平的实时多目标跟踪精度。综合精度和速度，这里我们选择了FairMOT算法进行人流量统计/人体检测。
优先运行C3F引用的那些代码
2021_SASNet（可运行）
-
代码地址——TencentYoutuResearch/CrowdCounting-SASNet ；数据集ShanghaiTech（拿到了）:https://drive.google.com/drive/folders/17WobgYjekLTq3QIRW3wPyNByq9NJTmZ9?usp=sharing
-
运行代码的方法：
# 1.先把SASNet_ROOT从github上下载下来，解压并重命名为SASNet_ROOT(这是作者规定的）
# 先准备好数据（Generating the density maps for the data:） # 原始数据默认放在 \home\teddy\UCF-QNRF_ECCV18，你得把数据放过去 # 整理好的数据存在F:\home\teddy\UCF-Train-Val-Test python prepare_dataset.py --data_path ./datas/part_A_final python prepare_dataset.py --data_path ./datas/part_B_final # 运行代码完成训练。注意用的是python这个命令而不是python3 python main.py --data_path ./datas/part_A_final --model_path ./models/SHHA.pth python main.py --data_path ./datas/part_B_final --model_path ./models/SHHB.pth
-
一开始是报错说cuda空间不够应该是加载了数据集和模型以后没空间训练了；
Traceback (most recent call last): File "main.py", line 105, in <module> main(args) File "main.py", line 81, in main pred_map = model(img) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_SASNet\SASNet_ROOT\model.py", line 173, in forward x1_density = self.density_head1(x1_out) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\nn\modules\container.py", line 117, in forward input = module(input) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_SASNet\SASNet_ROOT\model.py", line 239, in forward return torch.cat(outputs, 1) RuntimeError: CUDA out of memory. Tried to allocate 942.00 MiB (GPU 0; 4.00 GiB total capacity; 1.63 GiB already allocated; 0 bytes free; 3.05 GiB reserved in total by PyTorch)
（解决方案1：）怎么能够让读进cuda里面的图片和模型不那么大？比如修改batch_size,变得小一点，一次训练读进的图片少一点.main.py代码的这些位置，我尽可能改到最小了，但是还是cuda大小不够
#—————— 作者原来设定的batch_size是4，我GPU只有4个G，图片读进去以后，无法做训练计算了，所以我改成了1张 #—————— 但是改成了1张以后，还是GPU不够用，log_para和 parser.add_argument('--batch_size', type=int, default=1, help='batch size in training') # ——————log_para原来是1000，我改成了1 parser.add_argument('--log_para', type=int, default=10000, help='magnify the target density map') # ——————block_size原来是32，我改成了160 parser.add_argument('--block_size', type=int, default=160, help='patch size for feature level selection')
（解决方案2：）GPU不够大读不进去，那用CPU版本的pytorch做啊，他要求设定设备为CPU版，但是我不会设定（可以根据网上的教程设定一下，很简单的）
D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs: D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\numpy\.libs\libopenblas.NOIJJG62EMASZI6NYURL6JBKM4EVBGM7.gfortran-win_amd64.dll D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\numpy\.libs\libopenblas.QVLO2T66WEPI7JZ63PS3HMOHFEY472BC.gfortran-win_amd64.dll stacklevel=1) Traceback (most recent call last): File "main.py", line 112, in <module> main(args) File "main.py", line 69, in main model.load_state_dict(torch.load(args.model_path)) File "D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\torch\serialization.py", line 713, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\torch\serialization.py", line 930, in _legacy_load result = unpickler.load() File "D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\torch\serialization.py", line 876, in persistent_load wrap_storage=restore_location(obj, location), File "D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\torch\serialization.py", line 175, in default_restore_location result = fn(storage, location) File "D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\torch\serialization.py", line 152, in _cuda_deserialize device = validate_cuda_device(location) File "D:\Programing_File\Anaconda3\envs\paddle\lib\site-packages\torch\serialization.py", line 136, in validate_cuda_device raise RuntimeError('Attempting to deserialize object on a CUDA ' RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
（解决方案3：）用澳科大的超算或者点头的超算，cuda容量6G 8G应该就不会这样了
学校超算可以登录，但是我不会把自己的文件批量从本地传到服务器.
代码和数据集做成压缩包了，但是传不上去（一直显示在上传，但是一直没传上去）
学校的这个服务器，不允许我安装东西，修改环境,任何包都不允许我安装（目前在检查是否可以安装conda）
在购买的服务器上把所有数据放上去，就跑通了。训练的过程不显示训练进度，最后出一个分数
把本地代码改成CPU版本运行，尝试在本地运行
2018_CSRNet
2018_Cite=1004_CVPR_Li——CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes
官方实现：leeyeehoo/CSRNet-pytorch
上面这个代码运行有bug，解决方案：xr0927/chapter5-learning_CSRNet
其他人实现：
jupyter做的keras:RTalha/CROWD-COUNTING-USING-CSRNET
做了优化：karanjsingh/Improved-CSRNet
jupytert做的： krutikabapat/Crowd_Counting
jupyter：Neerajj9/CSRNet-keras
jupyter ： dattatrayshinde/oc_sd
教你怎么debug：Bazingaliu/learning_CSRNet
CS3244-AY2021-SEM-1/csrnet-tensorflow
1CS3244-AY2021-SEM-1/csrnet-pytorch
2016_MCNN
2016_CIte=1484_Zhang__Single Image Crowd Counting via Multi Column Convolutional Neural Network
官方：https://github.com/svishwa/crowdcount-mcnn
其他人实现：
https://github.com/CommissarMa/Crowd_counting_from_scratch/blob/master/crowd_model/mcnn_model.py
CS3244-AY2021-SEM-1/mcnn-pytorch
2021_P2PNet（数据）
（数据集整理的格式不对+放错位置了）
-
github上有其他人做的，用其他方式
-
数据集：（1）NWPU-Crowd（有）（用的这个） （2）SHTechPartA，PartB（有）（3）UCF_CC_50（有）（4）UCF_QNRF（有）————https://pan.baidu.com/s/1c2eLEE7leN0jz-fM38zyIQ ；——github地址：TencentYoutuResearch/CrowdCounting-P2PNet
-
卡在哪？报错信息：
他说num_sample=0所以报错。但是（1）我放在文件夹里面的数据集几百个，不是0个。（2）他这个num_sample的参数怎么设定，我一时半会也没从代码里看出来怎么回事--有可能是我数据集放的位置不对，他没数出我有多少张图片
P2PNet我猜原因是：
Traceback (most recent call last): File "train.py", line 222, in <module> main(args) File "train.py", line 123, in main sampler_train = torch.utils.data.RandomSampler(train_set) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\utils\data\sampler.py", line 104, in __init__ "value, but got num_samples={}".format(self.num_samples)) ValueError: num_samples should be a positive integer value, but got num_samples=0
原因找到了：这个位置、这个文件是空的所以报错“P2PNET_ROOT\new_public_density_data\shanghai_tech_part_a_train.list”。调用这个文件的代码文件是这个位置、这个文件“P2PNET_ROOT\crowd_datasets\SHHA\SHHA.py”。你需要了解调用的文件是什么格式的，才可以自己把“shanghai_tech_part_a_train.list”做出来并且让他可以运行
这两个文件我猜测应该是这样写的，配置完了，出了新的错误
新的报错是
初步看是由于python或者pytorch的版本问题而出错的，可以尝试把环境配置成和作者要求的一样试一试（我后面配置了和作者要求的版本完全一样的环境，还是报错，后面说了）
Start training Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> import torch File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> raise err OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies. Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> import torch File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> raise err OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies. Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path Traceback (most recent call last): pkg_name=pkg_name, script_name=fname) File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main mod_name, mod_spec, pkg_name, script_name) exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code Traceback (most recent call last): File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main Traceback (most recent call last): exec(code, run_globals) File "<string>", line 1, in <module> prepare(preparation_data) File "<string>", line 1, in <module> File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main import torch exitcode = _main(fd) _fixup_main_from_path(data['init_main_from_path']) exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main raise err Traceback (most recent call last): prepare(preparation_data) run_name="__mp_main__") prepare(preparation_data) OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll" or one of its dependencies. File "<string>", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main _fixup_main_from_path(data['init_main_from_path']) pkg_name=pkg_name, script_name=fname) _fixup_main_from_path(data['init_main_from_path']) exitcode = _main(fd) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main run_name="__mp_main__") mod_name, mod_spec, pkg_name, script_name) run_name="__mp_main__") prepare(preparation_data) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare pkg_name=pkg_name, script_name=fname) exec(code, run_globals) pkg_name=pkg_name, script_name=fname) _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path mod_name, mod_spec, pkg_name, script_name) import torch mod_name, mod_spec, pkg_name, script_name) run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path exec(code, run_globals) raise err exec(code, run_globals) pkg_name=pkg_name, script_name=fname) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll" or one of its dependencies. File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code import torch import torch mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code raise err raise err exec(code, run_globals) OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll" or one of its dependencies. OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll" or one of its dependencies. File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\train.py", line 7, in <module> import torch File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> raise err OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll" or one of its dependencies. Averaged stats: Traceback (most recent call last): File "train.py", line 228, in <module> main(args) File "train.py", line 167, in main args.clip_max_norm) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\engine.py", line 120, in train_one_epoch print("Averaged stats:", metric_logger) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\util\misc.py", line 186, in __str__ "{}: {}".format(name, str(meter)) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\util\misc.py", line 85, in __str__ median=self.median, File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\util\misc.py", line 64, in median return d.median().item() RuntimeError: median cannot be called with empty tensor
作者要求的环境：python=3.6.5，torch=1.5.0；我目前用的环境python=3.7.11,torch=1.7.1
我新开的了一个虚拟环境“py3.6.5_torch1.5”，python=3.6.5
确认python版本是不是3.6.5——是
然后装一个pytorch=1.5.0——是的，只不过是cpu版本的（一个电脑安装多个版本的cuda:https://www.cnblogs.com/yuyingblogs/p/16323438.html）
# 我的cuda版本是11.0；代码用了cuda，所以我得安装GPU版本的pytorch conda install pytorch==1.5.0 torchvision==0.6.0 cudatoolkit=11.0 -c pytorch
用这个新配置的环境运行了一下，有下面这个报错
显示你的程序读进去的是一个空的tensor，说明数据的位置还是不对，所以错了
Traceback (most recent call last): File "train.py", line 226, in <module> main(args) File "train.py", line 165, in main args.clip_max_norm) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\engine.py", line 120, in train_one_epoch print("Averaged stats:", metric_logger) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\util\misc.py", line 190, in __str__ "{}: {}".format(name, str(meter)) File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\util\misc.py", line 89, in __str__ median=self.median, File "F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\模型创新\2021_P2PNet\P2PNET_ROOT\util\misc.py", line 68, in median return d.median().item() RuntimeError: median cannot be called with empty tensor
是否考虑在数据集下面加一个scene文件夹，是不是文件夹格式不对造成的？——我尝试把文件整理成下面这样，但是运行了，还是报上面那个错误
train/scene01/img01.jpg train/scene01/img01.txt train/scene01/img02.jpg train/scene01/img02.txt ... train/scene02/img01.jpg train/scene02/img01.txt
我觉得上面出错应该是我用的运行命令不对造成的，你试试这个能不能运行
python train.py --data_root $DATA_ROOT \ --dataset_file SHHA \ --epochs 3500 \ --lr_drop 3500 \ --output_dir ./logs \ --checkpoints_dir ./weights \ --tensorboard_dir ./logs \ --lr 0.0001 \ --lr_backbone 0.00001 \ --batch_size 8 \ --eval_freq 1 \ --gpu_id 0
再去阅读代码，查看数据从哪里调用的？
train.py里面Line126的“train_set, val_set = loading_data(args.data_root)”我把args.data_root打印出来发现调用数据的地址是“./new_public_density_data”
看到这个位置，图片和ground truth要配对，是不是我没做好文件？
我极度怀疑跑不通的原因是：文件“shanghai_tech_part_a_train.list”里面应该一行一个图片名字一个ground truth文件的名字才能运行
C-3-Framework可以跑跑，据说各个数据集都跑通了
2022_CLTR（数据）
-
代码地址：dk-liang / CLTR；数据集： JHU-CROWD ++，NWPU-Crowd dataset
第一步整理数据集的时候就报错 python prepare_jhu.py --data_path /xxx/xxx/jhu_crowd_v2.0 [ WARN:[email protected]] global D:\a\opencv-python\opencv-python\opencv\modules\imgcodecs\src\loadsave.cpp (239) cv::findDecoder imread_('F:\FILES_OF_ALBERT\IT_paid_class\鐐瑰ご-姣曚笟璁烘枃\鍏紑鏁版嵁闆哱jhu_crowd_v2.0/test/images\0002.jpg'): can't open/read file: check file path/integrity F:\FILES_OF_ALBERT\IT_paid_class\点头-毕业论文\公开数据集\jhu_crowd_v2.0/test/images\0002.jpg Traceback (most recent call last): File "prepare_jhu.py", line 64, in <module> if img.shape[1] >= img.shape[0] and img.shape[1] >= 2048: AttributeError: 'NoneType' object has no attribute 'shape'
2020_M-SFANet（数据）
[M-SFANet] Encoder-Decoder Based Convolutional Neural Networks with Multi-Scale-Aware Modules for Crowd Counting (ICPR) [paper][code]
-
代码地址：https://github.com/Pongpisit-Thanasutives/Variations-of-SFANet-for-Crowd-Counting；数据集： Shanghaitech datasets (A&B)（拿到了） 、The Beijing-BRT dataset（有）
-
我猜原因是：数据文件夹没有按照他的要求放在他规定的位置，所以他拿不到数据，说你样本量为0，所以报错
File "train.py", line 60, in <module> trainer.setup() File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2020_M-SFANet\utils\regression_trainer.py", line 66, in setup for x in ['train', 'val']} File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2020_M-SFANet\utils\regression_trainer.py", line 66, in <dictcomp> for x in ['train', 'val']} File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\utils\data\dataloader.py", line 262, in __init__ sampler = RandomSampler(dataset, generator=generator) # type: ignore File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\utils\data\sampler.py", line 104, in __init__ "value, but got num_samples={}".format(self.num_samples)) ValueError: num_samples should be a positive integer value, but got num_samples=0
2019_BL（环境）
[BL] Bayesian Loss for Crowd Count Estimation with Point Supervision (ICCV(oral)) [paper][code]
github地址：https://github.com/ZhihengCV/Bayesian-Crowd-Counting；数据集：UCF-QNRF；
数据集放在了（F:\home\teddy），我不知道怎么样能指定我想存数据的地址
我觉得是你环境不符合要求导致的报错，报错都出现咋py37_torch171的包里面（这个多线程的包出了问题multiprocessing\spawn.py）——要不然就是我电脑配置不行，训练不起来；torch包本身也有问题（OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies.）
Traceback (most recent call last): Traceback (most recent call last): File "<string>", line 1, in <module> File "train.py", line 58, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) trainer.train() File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2019_BL\utils\regression_trainer.py", line 88, in train File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 225, in prepare self.train_eopch() _fixup_main_from_path(data['init_main_from_path']) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2019_BL\utils\regression_trainer.py", line 100, in train_eopch run_name="__mp_main__") File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 263, in run_path for step, (inputs, points, targets, st_sizes) in enumerate(self.dataloaders['train']): pkg_name=pkg_name, script_name=fname) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 96, in _run_module_code File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\utils\data\dataloader.py", line 352, in __iter__ mod_name, mod_spec, pkg_name, script_name) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\runpy.py", line 85, in _run_code return self._get_iterator() exec(code, run_globals) File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2019_BL\train.py", line 1, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator from utils.regression_trainer import RegTrainer File "F:\FILES OF ALBERT\IT_paid_class\点头-毕业论文\模型创新\2019_BL\utils\regression_trainer.py", line 6, in <module> return _MultiProcessingDataLoaderIter(self) import torch File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\__init__.py", line 117, in <module> File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\utils\data\dataloader.py", line 801, in __init__ raise err OSError: [WinError 1455] 页面文件太小，无法完成操作。 Error loading "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\site-packages\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies. w.start() File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__ reduction.dump(process_obj, to_child) File "D:\Programing_File\Anaconda3\envs\py37_torch171\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) BrokenPipeError: [Errno 32] Broken pipe
2022_GauNet（需要ubuntu环境）
（模仿了这个模型 DAU-ConvNet）
ubuntu环境，换了Ubuntu再运行
[GauNet] Rethinking Spatial Invariance of Convolutional Networks for Object Counting (CVPR) [paper][code]
-
代码地址：zhiqic/Rethinking-Counting ；数据集:QNRF,UCF50,SHHA,SHHB(我猜就是shanghai tech partA和B),UCF50]
-
配环境的过程
环境要求
Python=3.7（一开始说是3.5） tensorflow=1.13.1（这个版本tf适配的python是3.3-3.7） Ubuntu 16.04 (not tested on other OS and other versions) C++11 CMake 2.8 or newer (tested on version 3.5) CUDA SDK Toolkit (tested on version 8.0 and 9.0) BLAS (ATLAS or OpenBLAS) cuBlas
# 配置虚拟环境，python3.5 conda create --name py3.5_tf1.13.1 python=3.5 # python后的版本只允许写3.5，你写3.5.0就装不上了 从网站把whl文件下载下来，用pip安装————https://github.com/skokec/DAU-ConvNet/releases （安装whl文件会顺便把TensorFlow1.13.1下载下来，所以这一步之前，）
2019_SFCN（环境问题）
他要求python2.7 pytorch 0.5.,
But after installing py2.7, pytorch0.5 cannot be installed; the latest version of pytorch cannot be installed
The following specifications were found to be incompatible with each other: Output in format: Requested package -> Available versions Package vc conflicts for: torchvision==0.4.0 -> numpy[version='>=1.11'] -> vc[version='14.*|9.*|>=14.1,<15.0a0'] python=2.7 -> sqlite[version='>=3.30.1,<4.0a0'] -> vc[version='>=14.1,<15.0a0'] pytorch==1.2.0 -> cffi -> vc[version='14.*|9.*|>=14.1,<15.0a0'] cudatoolkit=11.0 -> vc[version='>=14.1,<15.0a0'] python=2.7 -> vc=9 Package zlib conflicts for: python=2.7 -> sqlite[version='>=3.30.1,<4.0a0'] -> zlib[version='>=1.2.11,<1.3.0a0'] torchvision==0.4.0 -> pillow[version='>=4.1.1'] -> zlib[version='>=1.2.11,<1.3.0a0'] Package vs2015_runtime conflicts for: cudatoolkit=11.0 -> vc[version='>=14.1,<15.0a0'] -> vs2015_runtime[version='>=14.15.26706|>=14.27.29016|>=14.16.27012'] cudatoolkit=11.0 -> vs2015_runtime[version='>=14.16.27012,<15.0a0'] Package cudatoolkit conflicts for: pytorch==1.2.0 -> cudatoolkit[version='>=10.0,<10.1|>=9.2,<9.3'] torchvision==0.4.0 -> cudatoolkit[version='>=10.0,<10.1|>=9.2,<9.3']