How long do I wait for newly introduced finches to settle down?

About an hour ago I introduced a new pair of finches to my very large cage (that already had an existing pair.) One of the finches that was originally in the cage is acting aggressively towards the new finches. So far the new guys are good at dodging the attacks, but I'm worried about when I am at school tomorrow and will not be able to watch them. Should I remove the aggressive finch right away and put him in a holding cage for a few days? Or should I let them duke it out for a while and then move him if things don't calm down? I just really don't want the little guys to hurt each other.