To test the potential of using amino acid abundances as a biosignature at icy ocean worlds, we investigate whether primordial amino acids (accreted or formed by early aqueous processes) could persist until the present time. By examining the decomposition kinetics of amino acids in aqueous solution based on existing laboratory rate data, we find that all fourteen proteinogenic amino acids considered in this study decompose to a very large extent (>99.9%) over relatively short lengths of time in hydrothermally active oceans. Therefore, as a rule of thumb, we suggest that if amino acids are detected at Enceladus, Europa, or other hydrothermally active ocean worlds above a concentration of 1 nM, they should have been formed recently and not be relicts of early processes. In particular, the detection of aspartic acid (Asp) and threonine (Thr) would strongly suggest active production within the ocean, as these amino acids cannot persist beyond 1 billion years even at the freezing point temperature of 273 K. Identifying amino acids from the oceans of icy worlds can provide key insight into their history of organic chemistry.