My understanding is that it is a), 5g of the complete formulation of A in B.
Assuming that you also took the complete formulation of B as 100%, of course.
If you take the API into account with the 10 ppm rule, I would expect strange values for the MACO at extreme changeovers including very high and very low dosage forms (I did not calculate any scenario, just a gut feeling).
The alternate “rule” - actually only an example given by PIC/S -, the 0.1 % min dose/max dose criterion, takes care of the “API-load” of your products.10 ppm is a simple arbitrary default value (without much of a rationale) that does not take into account the dosage of the API. It was probably thought as a correction where dose/dose considerations gave ridiculously high MACOs.
In my opinion and my experience, I consider only API as the contaminant. So, 5 mg of API can be allowed into next product.
Except API, other ingredients (excipients, colorants, WFI etc) cannot show pharmacological effect at lower concentrations. Hence, consider only API as the contaminant.