Find text graphics extent in axes data units

I need to show and hide labels on polygons, if the label too large for the polygon I need to hide it. If the label is smaller than the polygon, I show it again.

To do this I found a way that works some of the time, by using a text extent property.

            label = text(ax,0,0,'A','Visible','off');
            extent = get(label,'Extent');
            delete(label);
            
            % Convert Extent to area...
            extent_x = [0 extent(3) extent(3) 0];
            extent_y = [extent(4) extent(4) 0 0];
            characterAreaMapUnits = polyarea(extent_x,extent_y);

This seems to break down when I move the camera in a way that is too "3D," when this happens the extent property of the the text object is returned as [NaN NaN NaN NaN]. The documentation doesn't tell me that this can happen so I can't be sure what made it happen. Is there a reason this is happening? is there a better way to find the area of a text object?

This never seems to happen when I change the "Units" of the text object to anthing other than "data", but I need this in data units to properly compare it to the polygon that label is labelling.