MindSpore Insight Commands
Viewing the Command Help Information
mindinsight --help
Viewing the Version Information
mindinsight --version
Starting the Service
MindSpore Insight service only supports local access by default. For remote access, please modify the configuration file
mindinsight\conf\constants.py
. Modify theHOST
in the file to the server IP, and make sure that the startup port has been opened or the firewall has been closed.
mindinsight start [-h] [--workspace <WORKSPACE>] [--port <PORT>]
[--url-path-prefix <URL_PATH_PREFIX>]
[--reload-interval <RELOAD_INTERVAL>]
[--summary-base-dir <SUMMARY_BASE_DIR>]
[--enable-debugger <ENABLE_DEBUGGER>]
[--debugger-port <DEBUGGER_PORT>]
[--offline-debugger-mem-limit <OFFLINE_DEBUGGER_MEMORY_LIMIT>]
[--max-offline-debugger-session-num <MAX_OFFLINE_DEBUGGER_SESSION_NUMBER>]
Optional parameters are as follows:
Name |
Argument |
Description |
Type |
Default |
Scope |
Specifications |
---|---|---|---|---|---|---|
|
Optional |
Displays the help information about the start command. |
- |
- |
- |
- |
|
Optional |
Specifies the path for storing MindSpore Insight logs. |
String |
$HOME/mindinsight |
- |
- |
|
Optional |
Specifies the port number of the web visualization service. |
Integer |
8080 |
1~65535 |
- |
|
Optional |
Specifies the URL path prefix of the web visualization service. |
String |
Empty string |
- |
URL path prefix consists of segments separated by slashes. Each segment supports alphabets / digits / underscores / dashes / dots, but not single dot or double dots. |
|
Optional |
Specifies the interval (unit: second) for loading data. |
Integer |
3 |
0~300 |
The value 0 indicates that data is loaded only once. |
|
Optional |
Specifies the root directory for loading training log data. |
String |
./ |
- |
MindSpore Insight traverses the direct subdirectories in this directory and searches for log files. If a direct subdirectory contains log files, it is identified as the log file directory. If a root directory contains log files, it is identified as the log file directory. In the ModelArts development environment, this parameter can be specified as an OBS path. Please refer to ModelArts documentation for more information. |
|
Optional |
Whether to launch the MindSpore Insight Debugger. |
Boolean |
False |
True/False/1/0 |
The debugger entry can be shown on MindSpore Insight UI only when MindSpore Insight Debugger is launched. |
|
Optional |
Specifies the port number of the debugger server. |
Integer |
50051 |
1~65535 |
- |
|
Optional |
Specifies the maximum memory limit of a single offline debugger session. When the offline debugger cannot be executed due to insufficient memory, set it according to the device memory. |
Integer |
16*1024 |
6*1024~The upper limit of int32 |
- |
|
Optional |
Specifies the maximum session number of the offline debugger. The session number refers to the amount of training jobs that can be debugged at the same time. |
Integer |
2 |
1~2 |
- |
|
Optional |
Set the maximum number of graph nodes loaded by the debugger. |
Integer |
100000 |
1~2000000 |
- |
--workspace
log directory description:
Module name |
Log directory description |
Log format |
---|---|---|
datavisual |
Training kanban module, it records all the logs of training Kanban module. |
|
debugger |
Debugger module, it records all logs of the debugger module. |
|
explainer |
Explain the AI module, it records all logs that explain the data parsed by the AI module. |
|
gunicorn |
Web service module, it records all logs of the Web service module. |
|
lineage |
Traceability module, it records all logs of the traceability module. |
|
notebook |
Record all logs using MindSpore Insight in the ModelArts notebook. |
|
optimizer |
Optimizer module, it records all optimizer module logs. |
|
parse_summary |
Summary file parsing module, it records all logs when using the summary file parsing module. |
|
profiler |
Performance analysis module, it records all logs of the performance analysis module. |
|
restful_api |
RESTFul API module, it records all RESTFul API interaction logs. |
|
scripts |
Start and stop the MindSpore Insight module, it records all MindSpore Insight starts and stops. |
|
utils |
Public module, it records all logs of the public module. |
|
Note: There is one log file for each module, but when a log file exceeds 50M, it will be renamed and archived in the format of <module name>_<PORT>.log.<id>
,module name
indicates the module name, PORT
indicates the PORT number, and ID
indicates the number of file renaming and archiving times.
When the service is started, the parameter values of the command line are saved as the environment variables of the process and start with
MINDINSIGHT_
, for example,MINDINSIGHT_PORT
,MINDINSIGHT_WORKSPACE
, etc.
Execute command:
mindinsight start --port 8000 --workspace /path/to/workspace/dir --summary-base-dir /path/to/summary/base/dir
The startup is successful if it prompts as follows:
Web address: http://127.0.0.1:8000
service start state: success
Viewing the Service Process Information
MindSpore Insight provides user with web services. Run the following command to view the running web service process:
ps -ef | grep mindinsight
Run the following command to access the working directory WORKSPACE
corresponding to the service process based on the service process ID:
lsof -p <PID> | grep access
Output the working directory WORKSPACE
as follows:
gunicorn <PID> <USER> <FD> <TYPE> <DEVICE> <SIZE/OFF> <NODE> <WORKSPACE>/log/gunicorn/access.log
Stopping the Service
mindinsight stop [-h] [--port PORT]
Optional parameters are as follows:
Name |
Argument |
Description |
Type |
Default |
Scope |
Specifications |
---|---|---|---|---|---|---|
|
Optional |
Displays the help information about the stop command. |
- |
- |
- |
- |
|
Optional |
Specifies the port number of the web visualization service. |
Integer |
8080 |
1~65535 |
- |
Execute command:
mindinsight stop --port 8000
The shutdown is successful if it prompts as follows:
Stop mindinsight service successfully
Parsing Summary
MindSpore Insight provides tools for parsing summary log files. Users can save the scalars in the summary log file into a csv file and the images into a png file through the commands, which is convenient for viewing and further processing.
mindinsight parse_summary [--summary-dir] [--output]
Optional parameters are as follows:
Name |
Argument |
Description |
Type |
Default |
Scope |
Specifications |
---|---|---|---|---|---|---|
|
Optional |
Specifies the root directory of summary files. If the directory contains multiple summary files, only the latest summary file is parsed. |
String |
./ |
- |
The summary file directory needs to be readable and executable, and the summary file needs to be readable. |
|
Optional |
Specifies the root directory for saving output files. |
String |
./ |
- |
- |
Execute command:
mindinsight parse_summary --summary-dir ./ --output ./
The output directory structure is as follows:
└─output_{datetime}
├─image
│ └─{tag}_{step}.png
│
└─scalar.csv
In which,
output_{datetime} is the output directory. The rule is 'output_yyyyMMdd_HHmmss_SSSSSS' including year, month, day, hour, minute, second and microseconds.
{tag}_{step}.png is the image in training process. 'tag' and 'step' are the tag and step in the training (special characters in tag are deleted and '/' is replaced by '_').
scalar.csv is the file which save scalars (encoding: 'utf-8').
Using Mindoptimizer to Tune Hyperparameters
MindSpore Insight provides parameters tuning command. The command-line interface (CLI) provides the following commands:
usage: mindoptimizer [-h] [--version] [--config <CONFIG>]
[--iter <ITER>]
Optional parameters are as follows:
Name |
Argument |
Description |
Type |
Default |
Scope |
Specifications |
---|---|---|---|---|---|---|
|
Optional |
Displays the help information about the start command. |
- |
- |
- |
- |
|
Required |
Specifies the configuration file. |
String |
- |
- |
Physical file path (file:/path/to/config.yaml), and the file format is yaml. |
|
Optional |
Specifies the run times for tuning parameters |
Integer |
1 |
Positive integer |
- |