This has been reported before and is being looked into.  For the time being when you step away just select the speed until you return.  As a temporary fix when i see this happen i have copied the secondary flight plan and then activated the secondary flight plan and the issue goes away and my speed returns.