Release Notes

1.7.1 Release Notes

New Features

  • [STABLE] Support custom back propagation function for cell which uses parameters in the forward part.

  • [BETA] Support the SNN structure, and add demo on lenet and resnet50.

  • [STABLE] Support predicting the binding affinity network Pafnucy for protein-ligand complexes on Ascend.

Bug fixes

  • Fix the bug that the AICPU operator could not be found in some scenarios when the profiler parsed the performance data.

  • Modify the function of dying gasp, and save ckpt file name error bug.

  • Fix the derivative problem when the tuple input of an operator contains parameter. For example, the concat operator.

  • Fix the shard of expert bias in pipline with moe scenario.

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking.

Contributions of any kind are welcome!

MindSpore 1.7.0 Release Notes

Major Features and Improvements

OS

  • [STABLE] Support Python 3.8 (Linux/Windows/Mac).

  • [STABLE] Installation improved with more detailed install guide and automated shell scripts.

  • [STABLE] Support operator computing with multi-thread under Windows.

  • [STABLE] Compatible with GCC from version 7.3 to 9.x.

FrontEnd

  • [STABLE] Support dynamic weight decay for optimizers, that is weight decay value will change according to the increasing step during training.

  • [STABLE] Add four methods to create Tensor, which are mindspore.numpy.rand(), mindspore.numpy.randn(), mindspore.numpy.randint(), and mindspore.ops.arange().

  • [STABLE] Add mindspore.train.callback.History in Callback.

  • [BETA] Support custom operator implemented by Julia operator.

  • [STABLE] Support accessing attributes and methods of user-defined classes through mindspore.ms_class class decorator.

  • [STABLE] Support training when a network has side effect operations and control flow statements at the same time.

  • [STABLE] Support for more complex control flow syntax, such as a for loop statement in the body of a while loop.

  • [STABLE] Improve the performance of networks with complex syntax control flow statements by decreasing the num of subgraphs.

PyNative

  • [STABLE] Add Hook functions in PyNative mode, including register_forward_pre_hook, register_forward_hook of the forward hook interface, register_backward_hook of the reverse hook interface.

  • [STABLE] Optimize the execution performance of PyNative mode, and execute the front-end Python and the back-end C++ in parallel.

Auto Parallel

  • [STABLE] Support TopK routing, data parallel and optimizer state parallel when enable MoE.

  • [STABLE] Support AllGather/ReduceScatter communication operator fusion. Support AllReuduce fusion by the data volume size in DATA_PARALLEL mode.

  • [STABLE] Support ops.clip_by_global_norm in the parallel mode.

  • [STABLE] Support AdaSum optimizer in the parallel mode.

  • [STABLE] Support automatic optimizer state parallel.

  • [STABLE] Support AlltoAll configurable. Support automatically add virtualdataset cell.

  • [STABLE] Support automatically inference trainable parameters in pipeline parallel training.

  • [STABLE] Support clusters where the device number is not the power of 2.

  • [STABLE] Support sharding propagation in auto-parallel mode.

  • [STABLE] Support optimizer offload under the unified runtime.

  • [STABLE] Support Adafactor operator on CPU.

  • [STABLE] Support sharding at H/W axis for Conv2d/Conv2DTranspose operator. Support operators such as ResizeBilinear，ROIAlign, CropAndResize, BoundingBoxEncode, IOU and RandomChoiceWithMask.

Executor

  • [BETA] Failure Recovery Under Data Parallel Training Support auto failure recovery under data parallel training mode.

  • [BETA] Support searching for the number of threads under the CPU to obtain the optimal number of threads for execution. The entire search process takes 50 steps, and the overall performance will reach a stable state after 50 steps. When testing performance, data after 50 steps need to be used as a standard.

  • [STABLE] Enable concurrent executing on oneDNN for linux version. The performance of some CPU models may change a lot up or down.

DataSet

  • [STABLE] Add dataset operations mapping between TensorFlow.data module and MindSpore.dataset module, check list.

  • [STABLE] Python multiprocessing optimization and make processes exit normally.

  • [STABLE] Support Dataset Autotune for tuning the speed of dataset pipeline automatically.

  • [BETA] Dataset Offload support new data augmentation operations: RandomColorAdjust, RandomSharpness, TypeCast.

  • Output a single data column when __getitem__/__next__ methods of GeneratorDataset return a single NumPy object.

  • Use ulimit -u 10240 to increase the number of threads/processes available to the current user when specify too many processes or threads for loading dataset may cause RuntimeError: can’t start new thread.

API Change

Backwards Incompatible Change

Python API

  • Modify the gradient return value type of the hook corresponding to the register_backward_hook function, and change the gradient return value to the tuple type uniformly.(!31876)

  • Deprecated usage: import mindspore.dataset.engine.datasets as ds. Use import mindspore.dataset as ds instead as recommended in mindspore doc.

  • Add mindspore.ms_class interface, as class decorator for user-defined classes. It allows MindSpore to identify user-defined classes and access their attributes and methods(!30855)

  • Deprecate mindspore.SparseTensor and use mindspore.COOTensor instead. (!28505)

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking.

Contributions of any kind are welcome!