Once you find the greatest common factor, rewrite the expression as that factor times a set of parentheses. Then, for each term, think: "What do I multiply the GCF by to get this term?" That result goes inside the parentheses.

After you factor, your result should look like a coefficient times $a^m b^n$ times a binomial in parentheses. Compare both the outside factor and the terms inside the parentheses to each answer choice to find an exact match.

In Desmos, it is easiest to use $x$ and $y$ instead of $a$ and $b$. Type the original expression as

f(x,y) = 10x^5 y^3 - 15x^3 y^2.

For each option A–D, replace $a$ with $x$ and $b$ with $y$ and type it as a new function, for example g_A(x,y) = 5x^3 y^2 (2x^2 y - 3) for choice A, and similarly for the other choices.

For each choice, create a new expression that is f(x,y) - g_choice(x,y). Use sliders or a table to test several $(x,y)$ values (for example, $x=1,2$ and $y=1,2$). The correct choice will make f(x,y) - g_choice(x,y) equal to 0 for all tested values, while incorrect ones will give nonzero results for some values.

Look at the two terms: $10a^5 b^3$ and $15a^3 b^2$.

• For the numbers: the GCF of $10$ and $15$ is $5$.
• For $a$: the smallest exponent in the terms is $3$ (from $a^3$), so both terms share $a^3$.
• For $b$: the smallest exponent is $2$ (from $b^2$), so both terms share $b^2$.

So the GCF of the entire expression is $5a^3 b^2$. The expression can be written as

Now divide each original term by the GCF $5a^3 b^2$ to determine the terms that will appear inside the parentheses.

For the first term:

For the second term:

These quotients tell us the binomial's terms; keep the subtraction sign from the original expression for the final assembly in the next step.

Putting it together, we have

This exactly matches choice A) $5a^3 b^2 (2a^2 b - 3)$, so A is the correct answer.