This happens to me, too. I'm in the bike lane adjacent to heavy traffic and parked cars. A dude steps between the parked cars in front of me, looks right at me, waits until I'm about two feet from me then steps directly in my path. This happened twice last night. The first time, I barked, "Watch it!" and the guy jumped back. The second time it happened, I brushed the dude with my shoulder. He apologized, I kept going. Just to be clear, my brush was not intentional, although if I really tried I possibly could have avoided it.
I almost always give the benefit to the walker -- if somebody jaywalks in front of me, I'm okay with that. But you gotta at least give me some space to react.