"[Ben] seemed upset about it," a source tells PEOPLE of the incident involving Ben Affleck and Jennifer Garner's son Samuel
