are the hashtags treated together with the # char in ML part?
Issue #1
resolved
Check, what happens to hashtags? Is the # stripped off?
Comments (4)
-
-
- changed status to resolved
-
- changed status to open
-
reporter - changed status to resolved
We do not have any issue about this any more. I see hashtags are taken into account properly, as hashtags.
- Log in to comment
my_token_pattern=r"\w+(?:-\w+)+|[-+]?\d+[.,]?\d+|[#@]?\w+\b|[\U00010000-\U0010ffff]|[.:()[],;?!*]{2,4}"
This part captures the hashtags : [#@]?\w+\b