I've never seen any problems just having the filters available. It doesn't seem to cause any problems.
Many (maybe all) AE filters can't be used multi-threaded so combustion has to use them single threaded when rendering. This can create a bottleneck. But heck, AE filters are usually rendering bottlenecks anyway.