Leave for the winter, say
• Decline
• Deteriorate, slangily
• Migrate, maybe
• Remove chips surreptitiously from the table
• Start to tank
• Take a turn for the worse
• Turn bad
• Turn bad, slangily