Write an equation using the formula for the space diagonal of a rectangular prism, $d = \sqrt{l^2 + w^2 + h^2}$, with $d = 15$ and $l = 10$. Then square both sides to get a relationship between $w^2$ and $h^2$.

Write an equation from the surface area formula $S = 2(lw + lh + wh)$ using $S = 400$ and $l = 10$. Simplify to get an equation involving $w$, $h$, and $wh$.

Since the volume is $V = lwh$ and $l$ is known, you mainly need the product $wh$. Try introducing $p = wh$ and $s = w + h$, and use $(w + h)^2 = w^2 + h^2 + 2wh$ to connect your equations and solve for $p$ directly.

In Desmos, let $x$ represent $w$ and $y$ represent $h$. Enter the diagonal equation as x^2 + y^2 = 125 (this already comes from $10^2 + x^2 + y^2 = 15^2$).

Enter the surface area equation as 10x + 10y + x*y = 200. Desmos will plot both relations in the $xy$-plane.

Use the intersection tool (or tap where the curves cross) to find the intersection point with positive coordinates. Note the $x$- and $y$-values at this intersection and compute their product $x \cdot y$; this is $wh$.

In a new expression line, type 10 * x * y, substituting the intersection coordinates for $x$ and $y$. The resulting value is the volume of the prism and should match one of the answer choices.

Let the dimensions of the prism be length $l = 10$, width $w$, and height $h$.

For a rectangular prism, the space diagonal $d$ satisfies

We are told $d = 15$, so

Square both sides:

So

The total surface area $S$ of a rectangular prism with dimensions $l, w, h$ is

We are told $S = 400$ and $l = 10$, so

Divide both sides by 2:

Now you have two equations involving $w$ and $h$: equation (1) with $w^2 + h^2$, and equation (2) with $w$ and $h$ and their product $wh$. The goal is to find $wh$ because the volume is $V = lwh = 10wh$.

Let

• $p = wh$ (the product),
• $s = w + h$ (the sum).

From equation (1):

But

From equation (2):

so

Now substitute (4) into (3):

This becomes

Multiply both sides by 100:

Expand the left side:

Bring all terms to one side:

Solve this quadratic for $p$ using the quadratic formula.

From equation (5):

The discriminant is

so

Thus

So

• $p = \dfrac{600 + 500}{2} = 550$, or
• $p = \dfrac{600 - 500}{2} = 50$.

But $p = wh$, and we know from $(w - h)^2 \ge 0$ that

Using equation (1), $w^2 + h^2 = 125$, so

Therefore $p = 550$ is impossible, and we must have

The volume is

So the correct answer is 500.