Improve MEF compatibility

Issue #291 resolved
Alexey Yakovlev
created an issue

Hi Maksim,

I see that you mention MEF in #259 proposals, so I assume that you don't plan to abandon MEF extension. As a follow-up to this old thread —

https://bitbucket.org/account/notifications/read/40409634/great-project-direct-message

I'd like to discuss MEF compatibility issues. Looks like DryIoc offers the best out-of-the-box integration with MEF among other known containers. But still it's not 100%-compatible, so it cannot act as a drop-in replacement for most MEF-based applications. Here's my list (may be incomplete):

  1. For IEnumerable<T> and T[] dependencies, MEF uses ImportManyAttribute while DryIoc uses ImportAttribute.
  2. IPartImportSatisfiedNotification interface is not supported (OnImportsSatisfied callback method is not called when container completed its work).
  3. Custom MEF catalogs aren't supported.
  4. Recomposition isn't supported.

I myself don't use recomposition, so I only really care about points 1, 2 and 3 :)

Please let me know if you think these issues can be fixed at all. I would be glad to help with these, although I'll probably need your guidance.

Regards, Alexey

Comments (39)

  1. Maksim Volkau repo owner

    Hi Alexey,

    I would be interested in any help, so thanks for opening this issue.

    Mef support was the one of the major goals when I started to work on DryIoc. I don't care about recomposition, as it adds non DI related stuff into container.

    Regarding issues:

    1. ImportMany is not really unsupported, rather it is not required. But I probably miss something here. Will recheck. This is an easy one.

    2. PartSatisfied event requires the hook from DryIoc. Probably we can use RegisterInitializer for that now, but it may incline unnecessary perf penalty. Will check.

    3. Mef catalogs.. Ok, it needs to be implemented. Probably not so hard, just feed catalog discovered parts to Register or RegisterMany. As I remember, the directory catalogs may watch for dlls change, may be wrong here. But if so, we can hook them to UnknownResolver rule.

    It requires time from me. So I can check the points one by one. Or you help me. Any questions welcome.

    Btw, what about Mef2? Is it interesting for you?

    Thanks,
    Maksim

  2. Alexey Yakovlev reporter

    Thanks for your replies! I'd be glad to help, but I need to get familiar with the project internals first.

    I don't care about recomposition, as it adds non DI related stuff into container.

    Agreed!

    ImportMany is not really unsupported, rather it is not required. But I probably miss something here

    It's a question of compatibility. There is currently no way to declare an IEnumerable<T> or T[] import so it works both in MEF and DryIoc. In DryIoc, ImportMany T[] doesn't work, and in MEF, Import T[] results in an exception.

    Probably we can use RegisterInitializer for that now, but it may incline unnecessary perf penalty.

    I think it could be configured as an opt-in feature if it inclines performance penalty. OnImportsSatisfied notification is not needed for constructor injection, but it's very useful for property injection. There is no other way to initialize the component when all dependencies are satisfied.

    Probably not so hard, just feed catalog discovered parts to Register or RegisterMany.

    I'm not sure, in that case, who will be satisfying imports of the parts coming from that catalog? DryIoc will pull registrations out of the catalog and resolve dependencies while the catalog itself will only create component instances, right?

    As I remember, the directory catalogs may watch for dlls change, may be wrong here.

    They don't watch for changes themselves, one need to call DirectoryCatalog.Refresh to update the catalog.

    Btw, what about Mef2? Is it interesting for you?

    Not really. We have a rather large pool of Mef1 applications, and we considered porting them to Mef2 which has better perf numbers. But it looks like Mef2 is far less compatible with Mef1 than DryIoc.

    Regards, Alexey

  3. Maksim Volkau repo owner

    fixed: #292: Add ImportManyAttribute support for full MEF compatibility

    The other thing to improve compatibility which is still missing is #195: Composable Metadata as a IDictionary of string - object

    I plan to create the separate issues for the points mentioned.

  4. Maksim Volkau repo owner

    fixed: #294 Support for MEF IPartImportSatisfiedNotification interface in DI.Mef

    Thanks for PR, especially for the tests. I did change the registration for IPartImportSatisfiedNotification for compatibility and performance. Can be used with normal registrations as a bonus.

    Now, I can release what we have without ComposablePartCatalog support, may be as preview.

    For catalogs we need to convert Import/Export definition to ExportRegistrationInfo, which should be straightforward.

  5. Alexey Yakovlev reporter

    That's great!

    For catalogs we need to convert Import/Export definition to ExportRegistrationInfo

    But will it work for part types not available at registration time? I have two kinds of such scenarios: scripting (compile on resolve) and remoting (generating transparent proxies on demand).

  6. Maksim Volkau repo owner

    Hello again,

    Small update: I have implemented the Exporting of members without the need of containing type Export or even AsFactory attribute. Checking some small stuff before pushing.

    The next challenge beside the ComposablePartCatalogs (which I postponed a bit) is allowance of multiple ContractNames. Which is surprise for me. The idea for fixing this is to make ContractName a part of DryIoc metadata [#195], and changing the Import(Many) accordingly. That implies I need to break the current default behavior. I plan to make the changes to DryIoc 2.7 and DI.Mef 2.7, but enable the new ContractName behavior via opt-in flag. Then in V3 it may be changed to default.

  7. Alexey Yakovlev reporter

    Hello, that's amazing news!

    multiple ContractNames

    I'm not aware of that, what it looks like in code?
    Is that multiple exports? DryIoc already supports that:

    [Export(typeof(IFirstService))]
    [Export(typeof(ISecondService))]
    class MultipleExportService : IFirstService, ISecondService { }
    
  8. Maksim Volkau repo owner

    Here is many strings are exported with the same ContractName=Constants.SettingExportKey , then imported as ImportMany:

    // no Export attribute here
    internal class SettingProvider1
    {
        [Export(Constants.SettingExportKey)]
        private string ExportedValue { get; } = "SettingProvider1.ExportedValue";
    }
    
    // no Export attribute here
    internal class SettingProvider2
    {
        [Export(Constants.SettingExportKey)]
        protected string ExportedValue { get; } = "SettingProvider2.ExportedValue";
    }
    
    // no Export attribute here
    internal class SettingProvider3
    {
        [Export(Constants.SettingExportKey)]
        public string ExportedValue { get; private set; } = "SettingProvider3.ExportedValue";
    }
    

    Currently ContractName is mapped to DryIoc ServiceKey which should be unique per service type ("string" in this example) to uniquely identify the service.

  9. Maksim Volkau repo owner

    Hope you're well.

    I am still vac but did look into CustomComposableCatalog sample. The interesting part is the catalog itself creates value via GetExportedValue. I assumed (for some reason) the integration means to use catalogs as "type providers" for DryIoc registrations, and use for DryIoc for creating values. In case of TypeCatalog it possibly true, but for this Custom one it is different. And I don't know how to automatically differentiate.

    May be you can help. The way u creating actual proxy value is the only way, or it is possible to ask for proxy type and allow DI to create it? I've done similar things with Castle proxy.

  10. Maksim Volkau repo owner

    Question, what are the two first maradata fields in CustomComposablePartDefinition:

    public override IEnumerable<ExportDefinition> ExportDefinitions
            {
                get
                {
                    var metadata = new Dictionary<string, object>
                    {
                        { "ExportedInterface", ExportedInterface },
                        { "EnvironmentVersion", Environment.Version },
                        { CompositionConstants.ExportTypeIdentityMetadataName, ExportedInterface.FullName },
                    };
    
                    var export = new ExportDefinition(ExportedInterface.FullName, metadata);
                    return new[] { export };
                }
            }
    
  11. Alexey Yakovlev reporter

    Hi Maksim,

    I assumed (for some reason) the integration means to use catalogs as "type providers" for DryIoc registrations

    Catalogs serve as instance providers, not type providers. That's what my (a bit artificial) example demonstrates: you may get an object, but you may not get its type. Transparent proxies in .NET intercept the GetType method :)

    Roughly speaking, MEF deals with the catalog like this (pseudocode):

    // find the matching part definition(s)
    var partDefinition = catalog.Parts.Where(query)...;
    
    // create composable part
    var part = partDefinition.CreatePart();
    
    // set the required imports
    foreach (var importDef in partDefinition.ImportDefitions)
    {
        part.SetImport(importDef, exports); // resolve exports recursively
    }
    
    // create an instance
    part.Activate();
    return part.GetExportedValue(exportDefinition);
    

    And I don't know how to automatically differentiate.

    I don't think you have to differentiate (though you may just whitelist standard catalogs like TypeCatalog and AssemblyCatalog if you know how to deal with them efficiently).

    The way u creating actual proxy value is the only way, or it is possible to ask for proxy type and allow DI to create it?

    It's a simplified example taken from the real application. Transparent proxies are created by .NET runtime, and you cannot get their types and construct them using reflection like usual types. Actually, I don't think there is a way to reflect on transparent proxy type. Even the debugger doesn't show any internals of transparent proxies.

    Question, what are the two first maradata fields in CustomComposablePartDefinition:

    That's an example of custom metadata items. The only required metadata item is the last one, CompositionConstants.ExportTypeIdentityMetadataName.

  12. Maksim Volkau repo owner

    So yeah, whitelisting may be an idea. Then by default catalog can be registered via RegisterDelegate<ExportedType>, delagating to corresponding part GetExportedValue.

    The general question: what do you see as benefit in this case to use DryIoc instead of MEF itself?

  13. Alexey Yakovlev reporter

    The main reason is of course performance: MEF is very slow. Also, MEF performance is affected by various container options in a counter-intuitive way. MEF error reporting is awful, issue investigation often requires external tools like MEF explorer, etc. Another good reason is DryIoc extensions to MEF attributes, but that's another story :)

  14. Maksim Volkau repo owner

    Then you should be aware that creation of service by separate entity / catalog will decrease perf, cause DryIoc cannot optimize that. Especially if entity is the large graph. But in your case I think big chunk of perf will be eaten by proxy stuff.

    Then what's remaining for DryIoc to win over Mef:

    • caching of delagates
    • managing lifestyle in effective way
    • compose from both catalog and DryIoc registrations, where catalog provided value is just a some dependency in a graph.

    Whitelisting Type based catalogs effectively will turn them into normal DryIoc registrations, which is win.

    Summarizing: using catalogs as tool for type discovery will keep DryIoc perf. Otherwise, expect degradation.

    Btw: additional attributes will be ignored by Mef, until I somehow rediscover them from catalog data. So, yeah, another story.

  15. Alexey Yakovlev reporter

    Then you should be aware that creation of service by separate entity / catalog will decrease perf, cause DryIoc cannot optimize that. Especially if entity is the large graph.

    Sure, the custom catalog is a black box and it can perform poorly. But it won't construct the entire graph. It will only create an instance that is requested, and it won't satisfy its imports — that job is still done by DryIoc. So — yes, custom catalogs may decrease performance, but it will only affect composable parts taken from these custom catalogs.

    But in your case I think big chunk of perf will be eaten by proxy stuff.

    Proxies are slow, but network I/O they perform is times slower :)

    Then what's remaining for DryIoc to win over Mef:

    Wow, that's even better than I thought!
    You should consider rebranding DryIoc.MefAttributedModel as FastMEF or something :)
    To seduce folks looking for MEF-related nuget packages.
    It's far from obvious that DryIoc.MefAttributedModel can just replace MEF.
    RapidMEF, QuickMEF, SwiftMEF, SpeedyMEF, SmartMEF, JetMEF, you got the idea.

    BTW, another use case for DryIoc is to bridge MEF and MEF2. Modern libs like Roslyn use portable MEF2, and that's a pain in the ass for older MEF apps. But that's just an idea for the future, and seems like a lot of work.

  16. Alexey Yakovlev reporter

    Btw: additional attributes will be ignored by Mef, until I somehow rediscover them from catalog data. So, yeah, another story.

    Additional attributes perhaps can be converted to MEF metadata, so that they can be rediscovered from the catalogs trivially. But custom catalogs usually serve as a bridge between MEF and other technologies, and are likely to be populated from non-MEF sources. So, additional DryIoc attributes and custom MEF catalogs probably won't overlap.

  17. Maksim Volkau repo owner

    Hi again,

    I have at least returned back to the Mef part.

    Just implemented #323, started #325.. and stopped. I think I don't like it, multi contract names. The idea is very different from service keys.

    Actually, I see where it might be useful, but just to confirm. Did you use it this way?

  18. Alexey Yakovlev reporter

    Hi Maksim,

    Did you use it this way?

    I'm not sure whether it was a side project or just an experiment, but I don't see such a thing in my production code.

    Though I agree it can be useful, I think it's a marginally rare use case. So if it turns the code into a mess, I'm not sure it worth implementing.

  19. Maksim Volkau repo owner

    As we are closer to getting the fixes for compatiblity issues, I would like to decide on entry API methods.

    I think someone may not require MEF compatibility, and just use the attributes, so WithMefAttributedModel methods may proceed to live, providing the same rules as of current v2.5. The only addition will be WithDefaultReuseInsteadOfTransient.

    The full MEF compatibility may be provided via IContainer.WithMef including the modified and new wrappers, multiple contract names, etc.

    For someone who wants to customize, we may still provide initial Rules.WithMefRules. "Rules" suffix indicates that is not all support. Then provide the split methods for respective wrappers and other stuff support. Combining them together where it make sense.

    The other important thing is package name and version. At this point the extension may be renamed to DryIoc.Mef and the version may set to v3. Clearly there are breaking changes, but may be due rename we can set it to v1.

    What do you think?

  20. Alexey Yakovlev reporter

    I think someone may not require MEF compatibility, and just use the attributes

    It's hard to reason for me why would someone use MEF attributes and not use MEF wrappers at the same time. ImportMany and metadata attributes don't make sense without the wrappers supporting the metadata, and what's left is only enough for the toy apps.

    The other important thing is package name and version. At this point the extension may be renamed to DryIoc.Mef and the version may set to v3

    Well, it's up to you, I don't see anything bad here. DryIoc.Mef is clear and straightforward name, easy to find and remember, so that's just fine.

    But we're still far from full compatibility, at least while multiple exports #325, member metadata #385 and action exports #386 aren't supported. Perhaps something else pops up after these three... We can certainly leave custom MEF catalogs support out of scope for now, because it's extensibility stuff, advanced topics, hardly ever used by the majority of conventional applications.

  21. Maksim Volkau repo owner

    ImportMany and metadata attributes don't make sense without the wrappers supporting the metadata

    DryIoc has its own wrappers with metadata support. I think DI wrappers are more flexible and composable than predefined Mef variants. The probably only major difference, that ExportFactory disposes the dependencies, but I am not sure it is a good default behavior for everyone.

    Anyway, the idea is to provide some basis and allow user to add the features needed.

  22. Alexey Yakovlev reporter

    DryIoc has its own wrappers with metadata support

    Ok, I see. But I thought that you propose using MEF attributes and wrappers as a way to decouple an application from the container.

  23. Maksim Volkau repo owner

    There is no coupling when using DryIoc wrappers if you don't want to. The wrappers are standard .NET types, so you don't need to reference DryIoc.

    For instance close equivalent of MEF ExportFactory<T, TMetadata> is Tuple<Func<T>, TMetadata>.

  24. Maksim Volkau repo owner

    Now we have VS MEF, wich also cut on recomposition, and pre-serializing the object graph. It may provide a few ideas for how to substitute it for DryIoc.Mef. Not for this issue, but still promising direction.

  25. Log in to comment