mindformers.core

core module, including Runtime Context, Loss, Optimizer, Learning Rate, Callback, and Evaluation Metrics.

Runtime Context

mindformers.core.build_context	Build the context from config.
mindformers.core.get_context	Get context attribute value according to the input key.
mindformers.core.init_context	Initialize the context.
mindformers.core.set_context	Set context for running environment.

Loss

mindformers.core.CrossEntropyLoss

Calculate the cross entropy loss.

Optimizer

mindformers.core.AdamW	This is the implementation of AdamW.
mindformers.core.Came	Updates gradients by the Confidence-guided Adaptive Memory Efficient Optimization (Came) algorithm.

Learning Rate

mindformers.core.ConstantWarmUpLR	Constant Warm Up Learning Rate.
mindformers.core.CosineAnnealingLR	It has been proposed in SGDR: Stochastic Gradient Descent with Warm Restarts .
mindformers.core.CosineAnnealingWarmRestarts	Set the learning rate of each parameter group using a cosine annealing schedule, where \(\eta_{max}\) is set to the initial lr, \(T_{cur}\) is the number of epochs since the last restart and \(T_{i}\) is the number of epochs between two warm restarts in SGDR:
mindformers.core.CosineWithRestartsAndWarmUpLR	Cosine with Restarts and Warm Up Learning Rate.
mindformers.core.CosineWithWarmUpLR	Cosine with Warm Up Learning Rate.
mindformers.core.LearningRateWiseLayer	Learning Rate Wise Layer.
mindformers.core.LinearWithWarmUpLR	Linear with Warm Up Learning Rate.
mindformers.core.PolynomialWithWarmUpLR	Polynomial with Warm Up Learning Rate.

Callback

mindformers.core.CheckpointMonitor	Checkpoint Monitor For Save LossScale.
mindformers.core.EvalCallBack	Evaluate Callback used in training progress.
mindformers.core.MFLossMonitor	Monitor loss and other parameters in training process.
mindformers.core.ProfileMonitor	Profile analysis in training.
mindformers.core.SummaryMonitor	Summary Monitor can help you to collect some common information, such as loss, learning late, computational graph and so on.

Evaluation Metric

mindformers.core.EntityScore	Evaluates the precision, recall, and F1 score of predicted entities against the ground truth.
mindformers.core.EmF1Metric	Calculate the Em and F1 scores for each example to evaluate the model's performance in prediction tasks.
mindformers.core.PerplexityMetric	Perplexity is defined as the exponentiated average negative log-probability assigned by the model to each word in the test set.
mindformers.core.PromptAccMetric	Computes the prompt acc of each entity.