Attorney General Eric Holder attends the National Law Enforcement Officers Memorial Candlelight vigil event in May, 2013. Photo by James Tourtellotte/U.S. Customs and Border Protection (CBP)

Eric Holder has spent more than three decades working for the Department of Justice (DOJ). He was the first African American US Attorney General. In 2013. he led substantial changes in the prosecution of low-level drug offenders. The move is expected to ease the financial and societal burden of overpopulated prisons.

But the question remains: How will Holder be remembered by history?

Peter Schweizer, research fellow at Stanford University and author of "Throw Them All Out", says Holder broke a significant color barrier. However, there is one element of the Holder legacy that could be remembered above anything else: his failure to prosecute big banks.

In 2007, some of the world's largest banks, many of which were engaged in illegal or unethical mortgage sales, came to the brink of collapse and were saved by a government bailout. Many critics felt that while the government sought to prevent disaster, they failed to adequately punish criminal behavior.

"When you talk about criminal behavior you're talking about intentional fraud," Schweizer said.

The DOJ's position on banking crimes has been largely financially motivated instead of criminally, Schweizer said. Goldman Sachs, for example, was fined $550 million for the fraudulent sale of mortgage-backed securities, just four percent of the total amount the company paid in bonuses to its executives this year.

"If you want to have a deterrent effect on fraudulent behavior, you have to start putting people in jail," said Schweizer. "That's simply the way that it functions for the rest of us."

Schweizer also said that DOJ officials are often in a "revolving door" with banks. Officials often have stints with private law firms, where they represent or potentially represent the very banks they are later tasked with regulating.

In the gaps between increasingly high profile jobs at the DOJ, Holder worked as a private lawyer. From 2001 until President Barack Obama appointed him Attorney General in YEAR, Holder represented clients such as the National Football League, IBS, and Merck for Covington & Burling, a private law firm in Washington D.C.

"I think that's very very troubling when it comes to something like criminal justice," Schweizer said.

Related Stories