I googled what I think is your casting machine. A Kerr 2.5" flask dental casting machine. Here’s two thoughts.
If you’re only trying to add 1/2 inch bigger flask, you could alter this arm with a bit of steel, an angle grinder, hack saw and an arc welder. It doesn’t look that complicated for someone who knows how to weld. I could be wrong, but I’m pretty sure that a slightly bigger flask isn’t going to throw off your ability to balance the arm. Again, this is just a guess!
Yesterday, I also googled a replacement arm for this casting machine and I found some bigger arms that I think will fit it. Unfortunately, today I can’t find them. (I’m on another computer this morning). But I think you can buy a bigger arm to fit your casting machine. Rio Grande or similar can answer that question. The negative to that scenario is they were all expensive. $350ish.
Remember that if you change anything, you’ll have to check that everything still fits.
In answer to your first question about securing the table. Here’s an old Orchid thread asking the same question.
Keep us posted!
Jeff