Skill

kermt-add-cmim-pretrain

Created by nvidia

Version 392ab8ab2587345a9d86240967f666bd59ac82ca

Overview

Convert a grover_base checkpoint (encoder-only or encoder + vocab heads) into a hybrid checkpoint by adding a randomly-initialized cMIM decoder + latent_dist, then continue pretraining on the user's corpus as hybrid (vocab + contrast). Effectively kermt-continue-pretrain with a one-time ckpt-conversion step prepended.

Source/credit

All skills