Levy was born in Encino, California. Two of his uncles and one cousin played guitar, and his mother briefly took lessons. His grandfather, George Wyle, worked for The Andy Williams Show and The Flip Wilson Show on television as music director. As a teenager, Levy was a member of a local big band. After he graduated from high school, he studied at the Dick Grove School of Music, where his teachers included Ted Greene and Jimmy Wyble.[1]

He moved to San Francisco in 1990 and worked as a studio musician, appearing on a Tracy Chapman album in the mid 1990s. At the end of the decade, he was a member of the group Killer Joey with drummer Joey Baron, guitarist Steve Cardenas, and bassist Tony Scherr. Levy then moved to New York City, where he met Norah Jones and was a member of her band at the beginning of her career through her bestselling albums and world tours. He has also played with Rosanne Cash, Chris Difford, Amos Lee, and Lisa Loeb. He leads a trio called the Mint Imperials.[1]

Since 2015, Levy has been the Chair of the Guitar Performance department at Los Angeles College of Music.[2]