If you compare A vs B the genes's fold change will have the opposite sign to B vs A. So will be the gene set up or down-regulated depending on the comparison to take.
The gene set test analyze if a given group of elements is sorted in a certain way in a list (I am talking about the GSE like the one you performed or the one in the Broad Institute). Usually it is used to say that gene set X is up-regulated in the comparison A vs B, that is, gene set X (in general) is more expressed in A than in B. The enrichment is measured in a enrichment score (or a normalized enrichment score, the higher, the clearer trend/distribution group X has. To asses X is really differentially expressed a p-value is also calculated.
Despite that the number of the up-regulated gene sets sum up all the gene sets tested as pointed by juod, the number of gene sets with a a p-value below 5% and 1% is not the same in both comparisons which indicates that you are not really using the same samples in both gene sets.
Check how you performed your comparison. Are the genes in each comparison with the same value and opposite sign?
Check also the software used to run the GSEA analysis (if I recongize the type of report is from a package in Bioconductor [I don't recognize which one now]) as you (or me) might have misunderstood something or there is an error if the comparisons are done properly. For instance it could be that the report of p-values below the threshold would be from only the up-regulated gene sets.