Python 3 in myhdl 0.8.1

Issue #23 new
Jose M. Gomez
created an issue

I have tried to install myhdl 0.8.1 in python 3.4 as it is the one we use at work. I have found some syntactic issues that I have solved on my own, and I add the patch. Now my problem is that I have tried to test the installation with the HelloWorld example and I get this error:


ImportError Traceback (most recent call last) <ipython-input-21-dac7a6da6f3d> in <module>() ----> 1 from myhdl import Signal, delay, always, now, Simulation 2 3 def HelloWorld(): 4 5 interval = delay(10)

myhdl/init.py in <module>() 112 113 --> 114 from _bin import bin 115 from _concat import concat 116 from _intbv import intbv

ImportError: No module named '_bin'

Has anyone an Idea on how to solve this issue, as the _bin.py is already in the required directory.

Thanks in advance

Comments (36)

  1. Jose M. Gomez reporter

    I am trying to make the conversion. At least, to put into work the simulator. I have been upgrading schemes to python 3. I have found a problem with the sorting of the delays. I attach the error. Which value shall I use for the sorting?

    Thanks in advance

    myhdl/_Simulation.py in run(self, duration, quiet) 157 raise _SuspendSimulation( 158 "Simulated %s timesteps" % duration) --> 159 _futureEvents.sort() 160 t = sim._time = _futureEvents[0][0] 161 if tracing:

    TypeError: unorderable types: _DelayWaiter() < _Waiter()

  2. Jose M. Gomez reporter

    Patch to transform the myhdl 0.8.1 to python 3. It fulfills the core tests, after upgrading them also.

    There are some issues regarding verilog. In some web examples a while(1) appears instead of an always, I do not know if this is correct.

    Also the Cosimulation does not work, but this could be because I have little or no knowledge.

    Best,

  3. jandecaluwe repo owner

    For the myhdl codebase, I am not fond of an approach that immediately breaks compatiblity with Python2. The result is 2 codebases to maintain on separate branches, which are unnecessarily different. In the mean time, some non-trivial packages have managed to have a single codebase for both Python2 and Python3. I don't think this will be possible for all of myhdl (e.g. AST work) but in any case I think it's best to branch of as late as possible. There is a lot of low-hanging fruit by using 2.7 backports and future imports. I would strongly prefer a gradual approach, in which we modernize the code, one compatiblity feature at the time.

  4. Keerthan Jaic

    I'm working on python 3 support with a single codebase in my fork: https://github.com/jck/myhdl/tree/py3 I created a simple compatibility module(like pandas, flask etc), and I've been incrementally making changes while keeping an eye on travis to make sure py2.6,2.7 and pypy does not break. You can check the current test status here: https://travis-ci.org/jck/myhdl

    Feel free to send pull requests to that repo if any of you want to help out. I'll eventually create a PR here when all the modernization is done. After that we need to brainstorm about how to deal with division, unicode and nonlocals.

  5. jandecaluwe repo owner

    Keerthan - that looks just like what I'm looking for - great work!

    I am thinking of moving my MyHDL repo's to git and github. I notice you develop on github. How did/do you move between mercurial and git?

  6. Christopher Felton

    OT: @jandecaluwe and @Keerthan Jaic , my opposition on moving to git/github has diminished. The myhdl-bitbucket user base is small enough that a move will not be disruptive. With that said, I still have way more issues (failures) with git than hg (e.g. completely loosing workspaces). But if the main developer and the largest contributor have strong feelings on moving, we probably should just do it. Someone will need to take the time to explain merge vs. rebase so some idiot doesn't do both (not saying I have done this in the past :)

    Also note, there is a CI service/tool available for bitbucket, https://drone.io/.

  7. jandecaluwe repo owner

    Some remarks.

    I did an experimental conversion using hg-git: https://github.com/jandecaluwe/myhdl. It worked, including the bookmarks workaround for branches, and preserved contributor info, which I found amazing :-). However - changeset ids are different from yours, so direct incorporation of your work would not be possible.

    Regarding Py3K: I like your approach. I seems inspired by Armin Ronacher's work and incremental. I like it more than Jose Gomez' approach that seems to have a lot of inline tests. Also, I want pull requests or git/hg merges, not patches. But perhaps his work covers more and can be useful as reference. Did you review it?

    Seems we are diverging here. I would like to get 0.9 out asap now, based on mercurial/bitbucket or git/github, and then have a good basis for further development, with git/github. Any suggestions on how to proceed?

  8. Keerthan Jaic

    Firstly, before people start forking your repo, I would suggest renaming branch 'master' to 'stable', '0.9-dev' to 'master' and cleaning up:

    git branch -m stable
    git checkout 0.9-dev
    git branch -m master
    git merge stable
    git push -u --all origin
    

    And then delete the stale(merged) branches to avoid confusion.

    git push origin :mep107
    git push origin :0.8-dev
    git push origin :0.9-dev
    

    The result: https://github.com/jck/myhdl-1

    After this, I'll fork your repo, generate patches from my old repo, and apply it on the fork. Then, I'll send a pull request for 0.9, you enable travis CI and we work towards a release with a target date of a week or so.

  9. jandecaluwe repo owner

    Mm, I have an issue with this. I reviewed some git/github flows and it seems 'master' is always the "stable" branch. Often there is a 'develop' branch for development. This is similar to what I have now: 'default' has the releases + bugfixes, development is in a branch.

    Isn't it logical that most people would expect 'master' to be stable and not bleeding edge? Moreover, isn't it likely/desirable that we may have several development branches, e.g. for Py3K (like you have in your private fork)?

  10. Jose M. Gomez reporter

    Dear @jandecaluwe, if you do not like the inlined if's, I think that you can just delete the else clause and leave just the check at the beginning of the package. This will work the same way as the Armin Ronacher's recommendations, and maintain the string and long compatibility. I can do it, but please, confirm that it is what you really want. Second, I can make a push afterwards, but I need to know how do you prefer it, as a new branch or just in the 0.9-dev or whatever. Finally, I would like to know why the math operations always return an integer in the intbv class. Although this improves the calculations, it reduces the possibility to better mimic the HDL classes as information like the number of bits is lost.

  11. jandecaluwe repo owner

    @Jose M. Gomez What I want as a development flow is documented in great detail here: http://dev.myhdl.org/guide.html.

    Of course, people are free to ignore that, as you did, and try new things. No problem, I'm interested in good ideas and work in any form, and ready to adapt my flow for the better. However, don't expect me to apologize for confusion and redundant work that would have been avoided if my guidelines had been followed.

    In particular, if you had raised the py3k issue in the newsgroup first, there would have been a fruitful discussion and I'm pretty sure @Keerthan Jaic would have invited PR's on his work.

  12. Jose M. Gomez reporter

    Let's be pragmatic. Are you interested in the work I've done? If it is yes, do you want any modification before I try to commit it? If yes, which? If no, tell me if I shall commit it as a fork and from which version.

    If you have no interest, tell me and I will continue working on my own.

    Best

  13. jandecaluwe repo owner

    Being pragmatic is all I have been doing in this thread.

    I am obviously very interested in Py3K. I am also very worried about how that is done and very worried about a possible maintenance headache it may create. If I understand anything about Py3K, there are very good reasons to worry.

    I also notice there are 2 Py3K development efforts (that I know of). At this point, I like what I see from @Keerthan Jaic more, and he has a proven track record wrt MyHDL. I may be wrong. However, the coming weeks I have zero time to assess this further. I think he invited you to work together.

    In the short term, I want to get 0.9 out and possibly move to github. That will keep me busy more than enough. Only after that will I have time to assess and discuss Py3K again.

  14. Keerthan Jaic

    @jandecaluwe My main concern is with people often sending pull requests to the master branch. I'm not saying that there shouldn't be multiple development branches, just that there should be one branch which represents the latest and the greatest, and it should be obvious to any new contributor that pull requests should be sent to that branch. Since master is the default branch, any new clones will automatically use the master branch. End users will install from pypi or their distributions repo.

    If you want master to be stable, that's fine too. But, you must change the default branch on github to the development branch.

  15. Keerthan Jaic

    @Jose M. Gomez With your patch, I notice that all tests do not pass. Although it is definitely further ahead than my work.

    The main concerns with inline checks is that it makes the code uglier and might cause lots of repeated code.

    If you would like to work with me, please start sending pull requests to the 'py3' branch on github.com/jck/myhdl Try to make sure that each commit only takes care of a single detail. This makes it easier to revert particular changes and keep track of the test status. It also simplifies potential merge conflicts.

  16. Christopher Felton

    I do have one small request. Those (e.g. @Keerthan Jaic) familiar with git, could you take the time and outline some of the commands / operations needed to do some of this stuff. The MyHDL community has 6 years (or more) with hg. Any git pointers along the way would be greatly appreciated. Example, if I am not cloning through the github interface how would one clone the above and use the branch (and commit to only the branch).

  17. jandecaluwe repo owner

    @Keerthan Jaic I understand your reasoning and also found the rationale behind your original proposal. I have to think about it. From what I see, it seems that a workflow based on 'master' and 'develop' are much more common (and in line with what I did until now). Is that correct? Have you good examples of a 'master'/'stable' branch workflow?

    Regardless of the branch names - is it really so that new contributors should work on the latest development branch by default? Imagine a user who works with official releases, and now wants to contribute a bugfix. In my view, that should go against the "stable" branch, and merged later into the "development" branch. That is how I did it until now. Conversely, a longer-term developer is likely to understand that he should work on the "development" branch. Isn't that a good argument to make the "stable" branch the default, as a way to make things easy for the occasional bug fixer?

  18. Josy Boelen

    My two eurocents ...
    Wouldn't we better spend effort on extending MyHDL in stead of porting it to 3.4? I guess that for any RTL-engineer Python 2.7 is overwhelming enough? And 2.7 does the job. IMVHO porting to 3.4 doesn't really add value (other than being state-of-the-art Python? Please enlighten me if I'm wrong) where e.g. adding a library to package MyHDL modules for use in (my narrow field of view) FPGA-tools would truly benefit the user base. Another example: improving error reporting. I spent many an hour tracing the operation without a clue why it bombs out, to finally find out I left a trailing comma after a statement (cut-and-paste blues ...) The force of Python is in the libraries ( that's a quote on All Programmable Planet I remembered - Chris Felton?) so the force of MyHDL will be libraries ...
    About Git or Mercurial: they both are 'Swedish' to me (I think I can read that, but most of the time I'm just guessing ...) So I second Chris' request.
    The occasional bugfixer: that's what I do. I checked out the 0.9 'stable' branch and modify this locally. E.g. I added trace and log functions to (try to) understand what the Simulation and toVHDL were doing before that 'exception' pops up. I'll learn how to issue 'pull requests' (someday)
    Regards,
    Josy

  19. Christopher Felton

    Wouldn't we better spend effort on extending MyHDL in stead of porting it to 3.4? I guess that for any RTL- engineer Python 2.7 is overwhelming enough? And 2.7 does the job. IMVHO porting to 3.4 doesn't really add value (other than being state-of-the-art Python? Please enlighten me if I'm wrong) where e.g. adding a library to package MyHDL modules for use in (my narrow field of view) FPGA-tools would truly benefit the user base.

    It all depends and is complicated (how's that for skirting around the question :) There might be new users that walk away because there is no 3 support and eventually the project will need to go in that direction. Since this was user driven I don't see any need to thwart the development.

    For libraries/cores, I think MyHDL has the correct appoach as being one entry in the Python-HDL stack. MyHDL like Python is the foundation of the stack but I don't think we want fpga-tools integrated with MyHDL proper.

    The evolution of various tools will become the defacto over time, this is a good approach (e.g. our work with myhdl_tools which has evolved to gizflo but it is not my priority or focus, rather something I did quickly and shared).

    Another example: improving error reporting. I spent many an hour tracing the operation without a clue why it bombs out, to finally find out I left a trailing comma after a statement (cut-and-paste blues ...)

    It would be great if a commercial entity came along that used MyHDL and was willing to fund such an effort (not too exciting). I don't have any good ideas how to enhance this other than submitting small improvements as we run across them.

    The force of Python is in the libraries ( that's a quote on All Programmable Planet I remembered - Chris Felton?) so the force of MyHDL will be libraries ...

    I typically say "ecosystem" and yes, us users need to develop more of an MyHDL "ecosystem". For development and verification we benifit tremondously from the Python "ecosystem" but we don't have many cores (at this point) to leverage (lots of examples though). The biggest issue is that there are loads of folks experimenting / using but few package it up, some myhdl search results from September:

                     myhdl   
    github    [1]    1049    
    github    [2]    23   (repos)
    bitbucket [2]    46   (repos)
    bitbucket [3]    1360
    
    "myhdl fpga" on Google; About 9,270 results (0.31 seconds)
    
    [1] limiting to python sources
    [2] repositories only
    [3] google bitbucket domain search (searches all bitbucket)
    

    I'll learn how to issue 'pull requests' (someday)

    These are easiest through the web interfaces, it is a pretty slick process to contribute.

    Regards, Chris

  20. jandecaluwe repo owner

    @Josy Boelen If you have followed this thread you will notice that 1) I want to be very careful with Py3K 2) I am against "porting" in the sense of leaving Py2K behind, or having 2 codebases.

    My feeling is that the move towards Py3K will now finally accelerate. I also feel that we cannot delay doing something about Py3K much longer, or the project will suffer. However, my goal is to have a modernized, clean, single codebase that runs on both versions. I have on purpose waited a long time before taking action so that we can reuse experience from other projects, but my feeling is that the goal is now within reach.

    Yes, the power of Python is in the libraries. But Guido Van Rossum didn't write all of those :-), and I don't think that is the role of the core MyHDL project. (I even have no idea what a "standard" MyHDL library should look like.) Rather, I think we should expose the project to as many talented developers as possible, so that they are inclined to use, write and publish MyHDL libraries. Hence git, hence github, hence Py3K.

    One more note on Py3K: I think some design decisions do help to write cleaner code. But there is also one feature that I need to eventually realize my vision with MyHDL: nonlocals. We should have a single codebase and support it for a reasonable time, but at some point the future of MyHDL will be Py3K.

  21. Christopher Felton

    But there is also one feature that I need to eventually realize my vision with MyHDL: nonlocals. We should have a single codebase and support it for a reasonable time, but at some point the future of MyHDL will be Py3K.

    +1, nonlocals is a valid reason to move toward Py3k, exciting!

  22. jandecaluwe repo owner

    I see. What I like about the approach in those projects is that the "stable" branch is a maintenance branch named after the major release. So I propose to indeed use 'master' for development, but '0.8-maintenance' for bug fixes on the 0.8.* release. Whenever a new major release is out, I would create a maintenance branch accordingly. I think that should make it clear where pull requests should go.

    For the time being, I have named the repo 'myhdl-experimental' on github. I have renamed the branches as described above. I think it's ready to go, but please review it but don't fork it yet - suddenly I'm getting a lot of PRs :-). It took some juggling with branches and bookmarks. If it looks OK to you, I will rename the repo back to 'myhdl' on github + rename the bitbucket repo 'myhdl-retired' + send out a notice on the mailing list + adapt the design flow documentation.

    We should be there pretty soon, so you will be able to use git to transplant your commits. For my information - how will you go about doing that technically?

  23. Keerthan Jaic

    The approach sounds good! The repo looks good too. I just tried transplanting the commits by creating patches using git-format-patch and applying them to your repo using git-am. It worked fine.

  24. Jose M. Gomez reporter

    Dear @Keerthan Jaic,

    Thanks to @Christopher Felton I have been able to put into work my github repo. I have pushed all the changes that I have made to make myhdl work with py3. I have passed most of the tests. There are a few cases in the Verilog test bench that still fail. I have not been able to found the source, and some of them are also presente in the python2.7 case, which makes thinks a little bit more complicated.

    As I told previously, I am novice with git, so it is possible that some aspects are not properly done. If this is the case, please, tell me if you need some changes.

    Best

  25. Log in to comment