Hell yeah that's absolutely perfect. So in that case I'd suggest that they could have bumped into each other in the past two or so years - Byerly started doing more classic-type spy missions in Tevinter after he stepped down from Diplo head + they got eluvians set up in Minrathous.

So I propose that Cassian would know Byerly as Enrik, an unscrupulous and hedonistic merchant from Hercinia who had initially been in talks to smuggle silk to "Varian" as part of an entrapment operation. However, when they ended up meeting, they both got Bad Vibes from the other and ended up mutually breaking off contact because they were both trying to set the other person up, so no deal went through.