WASHINGTON – At the lowest moment in the highest law enforcement office, with criticism pouring in from all sides, including the president who appointed him, Atty. Gen. Alberto Gonzales made a rare reference last week to his difficult past, speaking defiantly of his determination to weather the controversy over the firing of eight federal prosecutors.

“Let me just say one thing,” Gonzales said. “I’ve overcome a lot of obstacles in my life to become attorney general. I am here not because I give up. I am here because I’ve learned from my mistakes, because I accept responsibility, and because I’m committed to doing my job.”

Gonzales has rarely spoken of the turmoil that has shadowed his family, emphasizing instead an inspirational biography that takes him from a boyhood in a cramped house that lacked hot water all the way to the elbow of a president.

The story is indeed impressive. Gonzales’ parents, Pablo and Maria, met as migrant farmworkers in Texas and settled in a town aptly named Humble. Pablo Gonzales worked in construction and later as a maintenance man. He was a hard drinker but a good provider, the story goes, who, with two brothers, built a two-bedroom house in which he raised Alberto and seven other children.

The reality, however, as reflected in public records and interviews, is grittier and more tragic. Gonzales’ family members have repeatedly stumbled, creating a bleak counterpoint to his dazzling rise to become the nation’s first Hispanic attorney general.

Gonzales’ father was arrested for drunken driving five times in 17 years covering much of Gonzales’ childhood and adolescence. Pablo Gonzales died in an industrial accident in 1982 when Gonzales was at Harvard Law School.

A younger brother, Rene Gonzales, died under mysterious circumstances in 1980. In 1991, the same year Alberto Gonzales became one of the first Hispanic partners at the white shoe Houston law firm of Vinson & Elkins, his younger sister Theresa pleaded guilty to possession of cocaine with intent to deliver. Nine years later, while Gonzales was on the Texas Supreme Court, his mother and another brother signed over their houses to a bail bondsman to raise bail for Theresa after she was charged with the same offense.

Most of these details did not arise in his Senate confirmation hearings, even though they might reasonably have been thought to affect his views about crime, drug and alcohol policy, and sentencing – all issues overseen or influenced by an attorney general.

Their omissions illustrate the remarkable extent to which Gonzales, 51, has managed to control the telling of his life story and the impenetrability of his outwardly mild and friendly manner.

They are also a function of Gonzales’ peculiar rise to power, an official whose career in government, first in Texas and then in Washington, has been under the protective wing of a single man. Since 1995, Gonzales has worked exclusively in jobs given to him by George W. Bush.

The current tempest is refocusing attention on this feature of Gonzales’ ascent. Some Democrats blame Gonzales’ loyalty to President Bush for his failure to challenge the firings of the prosecutors as politically and judicially questionable. And with many now calling for his head, Gonzales has no power base outside the Oval Office to protect him and lend support.

In addition, Gonzales’ sudden moment in the unflattering spotlight has prompted a re-examination of who he is as a person and about the background that propelled him to his exalted position.

The first son and second oldest of the Gonzales children, Alberto Gonzales from an early age seemed intent on advancing beyond Humble.

Gonzales attended Houston-area public schools and earned notice as an undersized but hard-hitting defensive back and linebacker for the MacArthur High School Exemplary Generals. He belonged to the National Honor Society, the International Club and the Christian Student Union.

“He surrounded himself with people he thought were going to be successful,” recalled Marine Jones, one of Gonzales’ guidance counselors.

But Gonzales learned early that friendship with whites was subject to painful challenges and he carries that memory to this day.

“I dated a girl in high school whose parents did not like me dating their daughter because I was Mexican,” Gonzales said in an unusually revealing address at the Air Force Academy last month. “She and I didn’t care about our differences, but her parents couldn’t see past them.”

“I remember vowing, at that time, to prove to the girl’s parents, someday, that I was just as good as they were,” Gonzales said.

In fact, he did just that, according to Teresa Osborne Brown, a MacArthur High classmate and former girlfriend.

Gonzales and Brown dated for part of their junior and most of their senior year, and attended the prom together.

But their courtship was bittersweet because, Brown said, her parents didn’t like their daughter dating a Hispanic.

“It’s pretty much a case of racial discrimination. It just didn’t set well with mom and dad,” said Brown, who now lives in Tennessee.

Years later, Gonzales, by then a partner at Vinson & Elkins, ran into Brown’s father on a Houston street. “He could have been vengeful, but he was nice to my father,” Brown said. “My father was very impressed. Dad had to admit he might have been wrong.”

Of his own father, Gonzales speaks almost worshipfully, marveling at a man “who worked harder than any person I have ever known.”

“I suppose to some he was just a common laborer with a 2nd-grade education; to me he was a hero with hands that could create anything,” he said in a speech last fall.

Gonzales acknowledged a darker side to his father. “My father had a terrible drinking problem,” Gonzales said in a June 2005 interview with the Academy of Achievement. “There were many nights when I remember him coming home and, you know, severe arguments with my mother and throwing the pillow over my head and just trying not to listen.”

Still, he said in the same interview, “no matter how much he drank on a particular night, if it was a work day the next morning, he was always up.”

From 1960 through 1977, records in Harris County, Texas, show that Pablo Gonzales was arrested five times for driving while intoxicated and convicted four times, the last time on a felony count.

He died at 52 in 1982 in an accident at the rice mill where he worked. Two years earlier, Gonzales’ younger brother, Rene, died violently in circumstances that are a tightly held within the family.

“He was killed back in “80 or “81 in Houston,” Gonzales’ brother, Antonio Gonzales, a Houston police officer, said in a telephone interview last summer. He also declined further comment.

By then Alberto Gonzales had gone farther than anyone in his family ever had. He’d graduated from high school, joined the Air Force, then won admission to the Air Force Academy.

After deciding he wasn’t suited to a military career, Gonzales transferred to Rice University in Houston, before moving on to Harvard for law school.

Gonzales spent much of the 1980s and early 1990s in a “dirt and deals” real estate practice at Vinson & Elkins, while building up his resume with service for civic groups like Catholic Charities, Leadership Houston and United Way.

Months after making partner at the law firm in 1991, Gonzales married his wife, Rebecca Turner Gonzales. Through service in the Texas State Bar Association, Gonzales met Harriet Miers of Dallas, who until January was White House counsel.

In the 1990s, Miers was Bush’s personal lawyer and helped bring Gonzales into the inner circle of the soon-to-be candidate for governor of Texas.

In the ensuing years, Gonzales has been a Bush retainer exclusively, serving as general counsel, Texas Secretary of State, Texas Supreme Court justice and White House counsel before taking the helm of the Justice Department in February 2005.

Bush took Gonzales to unimagined heights, but the rapid, unimpeded rise left him without much experience in making friends or in political street-fighting, both of which would come in handy now, said Glenn Smith, a former Democratic consultant in Texas.

“He was never in many public conflicts that he had to fight his way through,” Smith said. “We’ve never had a chance to take his measure. He is where he is because of Bush.”

In January 2000, while Gonzales sat on the Texas Supreme Court, his sister Theresa and an accomplice were arrested in a Houston-area drug bust in which police seized 40 grams of cocaine, nearly a pound of marijuana, more than $3,000 in cash, an assault rifle, a sawed off shotgun and a .50 caliber handgun, plus several hundred rounds of ammunition.

Court records indicate that at the time of her arrest, Theresa Gonzales was on 10 years probation following a guilty plea to a similar charge in 1991.

Gonzales’ mother, Maria Gonzales, 74, who still lives in the home her husband built, signed the property over to a bail bondsman to help bail out her daughter, according to records.

Court records show that Theresa Gonzales was sentenced to 90 days “jail therapy” and the charge against her was dismissed.

Theresa Gonzales, now 43, declined to comment. Justice Department spokesman Brian Roehrkasse said Alberto Gonzales didn’t know about his sister’s arrest until after the case was settled.

“The Gonzales family chose not to inform then-Judge Gonzales because of his position until after the matter had been fully adjudicated,” Roehrkasse said in a statement.

But now, perhaps for the first time, it is Alberto Gonzales who is the target of those who say he did something wrong and should be punished. So far, he has offered a vigorous defense, but that resolve will no doubt be tested as he faces his own form of public trial.