Mathematics - Circle Question with Solution | TestHub

Take $A=(1,0)$, $B=(0,1)$ and centre $(1,1)$. For the line $y=mx$, $|DE|=2\sqrt{2m/(1+m^2)}$ and the distance of $B$ from the line is $1/\sqrt{1+m^2}$, so the square of the area is proportional to $2m/(1+m^2{)}^2$, maximised when $m^2=1/3$. Hence $1/m^2=3$.