plaintiger: "score of finished games" is the score in previous, finished games (or matches) between you and your opponent in this game type. It doesn't concern the current match at all. Note that if you play a match of several games, it is counted as one game for all statistical purposes.

So 1:4 (=0) means that the player mentioned first has won 1 game, the second player has won 4 and no games have been drawn. When your current match is over, one of those numbers will go up by 1.