Tricks in MxNet & Gluon

Posted on | In | 0 comments |

Gluon

报错:

1
RuntimeError: Parameter 'stn_conv0_weight' was not initialized on context gpu(0). It was only initialized on [gpu(0)].

解决方法:

需放在 ```gluon.Trainer(net.collect_params(), opt.optimizer)``` 的前面.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16


## 提取特征层输出

### Method 1: 继承 Block, 实现 forward

... 待续

### Method 2: 使用 SymbolBlock

```python
net = gluon.model_zoo.vision.densenet(pretrained=True, ctx=ctx)
internals = net.load_params("./densenet.params", ctx=context)
out_list = [internals['densenet0_stage3_conv13_fwd_output'],
                      'densenet0_stage3_conv13_fwd_output']]
net = gluon.SymbolBlock(out_list, data, params=net.collect_params())

网络冻结

  • 方式一： freeze 层在在 record 外面， 只 forward, 梯度不进行回传

不同层设置不同的学习率

1
2
3
net = gluon.model_zoo.vision.densenet(pretrained=True, ctx=ctx)
for name, params in net.features.collect_params().items():
    params.lr_mult = 0.1

所设定层的学习率变为 base_learning_rate * params.lr_mult

梯度截断

1
gluon.Trainer(net.collect_params(), 'sgd', {'lr': 1e-2, 'grad_clip': 2})

MXNet

API

reshape

1
2
3
4
5
6
7
8
9
mxnet.ndarray.reshape(data=None, shape=_Null, reverse=_Null, target_shape=_Null, keep_highest=_Null, out=None, name=None, **kwargs)
"""
Some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below:
0 copy this dimension from the input to the output shape.
-1 infers the dimension of the output shape by using the remainder of the input dimensions keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1.
-2 copy all/remainder of the input dimensions to the output shape.
-3 use the product of two consecutive dimensions of the input shape as the output dimension.
-4 split one dimension of the input into two dimensions passed subsequent to -4 in shape (can contain -1)
"""

Debug

1
2
3
4
5
错误提示:
ValueError: You created Module with Module(..., data_names=['data']) but input with name 'data' is not found in symbol.list_arguments(). Did you mean one of: data0

解决方案:
mod = mx.mod.Module(symbol=sym, context=ctx, data_names=('data0',), label_names=None)
1
2
3
4
5
6
错误提示:  
DeferredInitializationError: Parameter disnet0_conv5_weight has not been initialized yet because initialization was deferred. Actual initialization happens during the first forward pass. Please pass one batch of data through the network before accessing Parameters. You can also avoid deferred initialization by specifying in_units, num_features, etc., for network layers.

解决方案:

在net的class里的init部分定义了层，在forward没有使用。删掉就好了