|  | 
 
 楼主|
发表于 2024-3-16 23:27:03
|
显示全部楼层 
| 训练点击后出错了,这是什么情况? Error: 2 root error(s) found.
 (0) Resource exhausted: OOM when allocating a buffer of 205521156 bytes
 [[node src_dst_opt_1/random_uniform_8/RandomUniform (defined at F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\framework\ops.py:1762) ]]
 Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
 
 [[concat_6/concat/_121]]
 Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
 
 (1) Resource exhausted: OOM when allocating a buffer of 205521156 bytes
 [[node src_dst_opt_1/random_uniform_8/RandomUniform (defined at F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\framework\ops.py:1762) ]]
 Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
 
 0 successful operations.
 0 derived errors ignored.
 
 Original stack trace for 'src_dst_opt_1/random_uniform_8/RandomUniform':
 File "threading.py", line 884, in _bootstrap
 File "threading.py", line 916, in _bootstrap_inner
 File "threading.py", line 864, in run
 File "F:\DeepFaceLab\_internal\DeepFaceLab\mainscripts\Trainer.py", line 58, in trainerThread
 debug=debug)
 File "F:\DeepFaceLab\_internal\DeepFaceLab\models\ModelBase.py", line 193, in __init__
 self.on_initialize()
 File "F:\DeepFaceLab\_internal\DeepFaceLab\models\Model_SAEHD\Model.py", line 327, in on_initialize
 self.src_dst_opt.initialize_variables (self.src_dst_trainable_weights, vars_on_cpu=optimizer_vars_on_cpu, lr_dropout_on_cpu=self.options['lr_dropout']=='cpu')
 File "F:\DeepFaceLab\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py", line 44, in initialize_variables
 lr_rnds = [ nn.random_binomial( v.shape, p=self.lr_dropout, dtype=v.dtype) for v in trainable_weights ]
 File "F:\DeepFaceLab\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py", line 44, in <listcomp>
 lr_rnds = [ nn.random_binomial( v.shape, p=self.lr_dropout, dtype=v.dtype) for v in trainable_weights ]
 File "F:\DeepFaceLab\_internal\DeepFaceLab\core\leras\ops\__init__.py", line 206, in random_binomial
 random_ops.random_uniform(shape, dtype=tf.float16, seed=seed) < p,
 File "F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\ops\random_ops.py", line 245, in random_uniform
 rnd = gen_random_ops.random_uniform(shape, dtype, seed=seed1, seed2=seed2)
 File "F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\ops\gen_random_ops.py", line 860, in random_uniform
 name=name)
 File "F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
 op_def=op_def)
 File "F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
 return func(*args, **kwargs)
 File "F:\DeepFaceLab\_internal\python-3.6.8\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3371, in create_op
 attrs, op_def, compute_device)
 | 
 |