I figured out why a model specification worked in fitglm, but not in fitglme even without random effect. Many others also had the same problem of rank deficit error in Fitlme and their questions were not answered.
The problem is that glmfit and lmfit drop the problematic terms automatically but glmfite and lmfite do not.
So the solution will be to run the same model specification without random effects in glmfit or lmfit and see if it works. If this is the case, then at least one term will have been dropped whose estimate is 0 and whose tstat and pValue equal NaN.
That term is the problematic one and you need to drop it manually before running fitglme or lmfite.