Additionally, a large limiting factor during SE's launch period was the lack of SKSE (The Skyrim script extender, used for many, if not most, mods) which has since released for SE. However this may be becoming less and less common considering owners of the Legendary Edition (Which was generally required for most large mods before the release of SE) received free upgrades to the SE when it first released. If there is an easy way to make them compatible, numerous authors choose to release and update versions for both.
The article linked by Kyle Rone in his comment on your post does offer some reasons why the versions aren't compatible, but is not technically an answer to your question, since it is the other way round. It would really depend on the mod (some could work 'out of the box' or with light modifications), but generally it is safe to assume SE mods won't work as intended for the Legendary Edition or any other version of original Skyrim.