[4:22:21 PM] Drew Roos: my last comment: uiVersion had value, but the main failure was no one remembered to update it. if version was really "[version].[uiVersion]", that's good in that upgrading the 'content version' must reset the uiVersion
[4:23:02 PM] Drew Roos: so free-form version allows us the preserve the benefit of version/uiVersion without the faults
[4:23:53 PM] Anton de Winter: Taking a snapshot of the conversation so far.
[4:25:19 PM | Removed 4:25:25 PM] Anton de Winter: This message has been removed.
[4:25:44 PM] Jonathan Jackson: @Cory - would you ever recommend we not use just an int?
[4:27:53 PM] Cory Zue: yes, i think if the formdesigner was smart enought to generate "[version].[uiVersion]" that would be an improvement
[4:28:09 PM] Jonathan Jackson: so you would actually spend the time to implement that in vellum?
[4:28:22 PM] Anton de Winter: That would be a 30 minute fix
[4:28:26 PM] Anton de Winter: (or less )
[4:28:30 PM] Clayton Sims: ...
[4:28:31 PM] Clayton Sims: "nope"
[4:28:32 PM] Jonathan Jackson: to know if something constitutes a model change?
[4:28:34 PM] Jonathan Jackson: not at all.
[4:28:38 PM] Jonathan Jackson: how about if i change branching logic?
[4:29:01 PM] Jonathan Jackson: or logic the nulls something out
[4:29:05 PM] Anton de Winter: vellum inherently knows  when you alter the model, or just text
[4:29:08 PM] Anton de Winter: for example.
[4:29:15 PM] Cory Zue: i can also imagine a world where you just want uids and want them to be a reference to all kinds of other metadata
[4:29:20 PM] Clayton Sims: So I hve a question WRT uiVerison/Version
[4:29:34 PM] Mitch Sundt: I would recommend staying with a single number.  The critical factor is what the server (aggregator) can do with the multiple versions.
[4:29:36 PM] Clayton Sims: Can you guys give me an idea of a pure "ui change"?
[4:29:39 PM] Cory Zue: and uids are easier to deal with than incrementing numbers, particularly in disconnected environments
[4:29:51 PM] Matt Adams: That's true.
[4:30:00 PM] Clayton Sims: Because if I change the text of a question from 
"Are you sick"
to
"Are you not sick"
that's not a ui change
[4:30:03 PM] Clayton Sims: that's a model change
[4:30:04 PM] Mitch Sundt: Each form builder will not be able to understand the limitations of the various aggregators and their abilities to combine datasets.
[4:30:19 PM] Matt Adams: Also true.
[4:30:23 PM] Mitch Sundt: That's why it should be either input by people or kept as a simple integer.
[4:30:31 PM] Cory Zue: clayton, i think the defintition being considered is "something that would cause a sql database to change it's schema"
[4:30:37 PM] Cory Zue: i agree it's not a very useful definition
[4:30:43 PM] Jonathan Jackson: its not useful at all.
[4:30:44 PM] Clayton Sims: So building an automated tool that "detects" model changes seems irrelevant, since we can't merge them correclty anyway
[4:30:49 PM] Clayton Sims: QED: Single integer
[4:31:00 PM] Cory Zue: here's what i consider a valid use case
[4:31:12 PM] Cory Zue: i make a form and deploy it for my pilot
[4:31:19 PM] Jonathan Jackson: the way you would test if there was a schema change is by regening the schema code off the xform, regardless of thre id, and checking if it matches
[4:31:25 PM] Cory Zue: then i iterate on it for a while and go through a bunch of versions
[4:31:26 PM] Drew Roos: a UI change: "formatting or phrasing change that does not meaningfully alter the question"
[4:31:32 PM] Cory Zue: then i deploy a totally new app
[4:31:35 PM] Clayton Sims: Drew: Undetectable
[4:31:38 PM] Cory Zue: and i want a simple way to say "was this a pilot form"
[4:31:46 PM] Clayton Sims: change the xmlns
[4:31:51 PM] Jonathan Jackson: or change teh version
[4:31:56 PM | Edited 4:32:00 PM] Cory Zue: that is readable to a human who doesn't have to remember the version number the change happened
[4:32:02 PM] Drew Roos: think of it like the 'minor change' checkbox on confluence
[4:32:15 PM] Clayton Sims: Have you _ever_ checked the "minor change button"?
[4:32:15 PM] Cory Zue: it's the same xmlns
[4:32:20 PM] Drew Roos: all the time
[4:32:23 PM] Clayton Sims: ...
[4:32:24 PM] Cory Zue: i want to see the data together in ~all places
[4:32:28 PM] Drew Roos: i make a lot of minor changes
[4:32:29 PM | Edited 4:32:31 PM] Cory Zue: but it's "importantly different"
[4:32:45 PM] Jonathan Jackson: but its only importantly different to you because of context
[4:32:51 PM] Anton de Winter: @Cory, in that example you'd still not know if 2.3 is the latest version (was there a 2.4?)
[4:32:56 PM] Cory Zue: i just don't understand when a less forgiving spec for an identifier was an advantage
[4:33:07 PM] Mitch Sundt: Cory: if you change 3 things, would you have "pilot+changes to foobar questions to fix spelling and changes to blah to add new items and changes to bobo to look for images from set" ?
[4:33:09 PM] Cory Zue: @anton i don't care
[4:33:09 PM] Drew Roos: seriously
[4:33:10 PM] Jonathan Jackson: less choices == easier for anyone coming along next
[4:33:14 PM] Cory Zue: i only care about 2/3
[4:33:20 PM] Mitch Sundt: How long would the string get?
[4:33:24 PM] Drew Roos: server interpret the version attr best the can, and degrade gracefully
[4:33:34 PM] Cory Zue: who cares?
[4:33:35 PM] Mitch Sundt: At some point, you need to have a short-hand representation of the change
[4:33:39 PM] Drew Roos: if it's an int, great, but if not, all bets are off and do a string compare
[4:33:41 PM] Mitch Sundt: Integer == shortest possible
[4:33:55 PM] Cory Zue: what is the argument for short?
[4:34:16 PM] Cory Zue: readable >> short
[4:34:20 PM] Cory Zue: freedom >> restrictions
[4:34:23 PM] Drew Roos: i think it's funny that the ODK guys, who want to be able to add custom attributes to everything, are so insistent on locking down this particular attribute
[4:34:48 PM] Mitch Sundt: And having to allocate 65,000 characters to hold a version string -- where does it stop
[4:34:50 PM] Drew Roos: if you don't understand a weird version format, ignore it
[4:34:57 PM] Jonathan Jackson: cory, the whole point of this is to restrict freedom
[4:34:59 PM] Jonathan Jackson: so I don't agree
[4:35:04 PM] Cory Zue: not for this attribute
[4:35:09 PM] Jonathan Jackson: yes,
[4:35:12 PM] Jonathan Jackson: for this attribute
[4:35:12 PM] Mitch Sundt: how long is long enough
[4:35:18 PM] Jonathan Jackson: we are saying no unicdoe, certainly
[4:35:18 PM] Cory Zue: _who cares_?
[4:35:24 PM] Yaw Anokwa: the odk guys want things that are simple. version=1/2/3 is simple. attributes to change appearance without whitelisting is simple.
[4:35:29 PM] Cory Zue: are you worried about running out of disk space storing your version ids?
[4:35:46 PM] Cory Zue: "put something unique here" is also simple
[4:35:54 PM] Mitch Sundt: MySQL has a 65536 byte limit on the size of any data row
[4:35:56 PM] Jonathan Jackson: put something here that is in ascii you mean?
[4:36:10 PM] Cory Zue: but your implementation can use ints
[4:36:14 PM] Mitch Sundt: That's 16,000 UTF-8 characters.
[4:36:21 PM | Edited 4:36:26 PM] Cory Zue: i don't see why you'd feel the need to force that on everyone else
[4:36:38 PM] Mitch Sundt: The standard is for cross-operation
[4:36:41 PM] Jonathan Jackson: we're only forcing the solution everyone is already going to go with
[4:36:48 PM] Mitch Sundt: The standard defines the minimal common subset.
[4:36:53 PM] Drew Roos: pretty much the only operation we'd ever do on this value is 'determine if one version is newer than another', right?
[4:36:58 PM] Jonathan Jackson: yes
[4:37:04 PM] Mitch Sundt: Your tools can support more, but they need to support at least this.
[4:37:07 PM] Jonathan Jackson: and as far as any of our programmatic capabilites are concerned
[4:37:12 PM] Jonathan Jackson: that means "did anything change anywhere on this form"
[4:37:26 PM] Cory Zue: (switch to postgres?)
[4:37:30 PM] Mitch Sundt: The impact is on the form creation tool.
[4:37:31 PM] Cory Zue: (kidding)
[4:37:42 PM] Mitch Sundt: PostgreSQL is definitely better.
[4:37:49 PM] Drew Roos: why not use timestamps then
[4:38:17 PM] Mitch Sundt: Timestamps would work.  But I think they aren't as user friendly as an integer.
[4:38:33 PM] Jonathan Jackson: correct
[4:38:38 PM] Drew Roos: and no one will begrude you if you server throws an error if someone tries to encode lord of the rings as xsd:base64binary in the version attribute
[4:38:51 PM] Drew Roos: monotonically increasing ints aren't really that user friendly!
[4:38:55 PM] Drew Roos: they're for machines, not people
[4:39:08 PM] Jonathan Jackson: that's not at all true.
[4:39:14 PM] Jonathan Jackson: everyone understand this:
[4:39:22 PM] Jonathan Jackson: "What version did the doc send us?  2"
[4:39:27 PM] Jonathan Jackson: "crap, we have 1, can you send me 2"
[4:39:40 PM] Cory Zue: that's my argument, jon
[4:39:42 PM] Yaw Anokwa: moreover, timestamps don't work well across time zones.
[4:39:42 PM] Mitch Sundt: If you want strings, spec a max size, but realize that 4 bytes for integers is way less burdensome on the backend storage than long strings
[4:39:56 PM | Edited 4:40:44 PM] Drew Roos: "what's the meaningful difference between 562 and 563"
[4:40:06 PM] Jonathan Jackson: NO ONE IS GETTING TO 562
[4:40:13 PM] Cory Zue: are you kidding me? of course they are?
[4:40:16 PM] Drew Roos: that's absolutely not true
[4:40:22 PM] Jonathan Jackson: who is getting to 562?
[4:40:25 PM] Drew Roos: every punctuation change in vellum is a new version
[4:40:31 PM] Cory Zue: anyone who uses a ui tool and presses save a lot
[4:40:38 PM] Jonathan Jackson: that's the designers choice
[4:40:40 PM] Mitch Sundt: You will always need a mapping between a short-hand and what was changed.
[4:40:49 PM] Jonathan Jackson: and in that case, you better not ever think someone is going to look at the version
[4:40:51 PM] Jonathan Jackson: so it doesn't matter
[4:41:12 PM] Cory Zue: so is the only real argument against the "anything" plan the space limitations on disk?
[4:41:25 PM] Jonathan Jackson: for me its just simplicity
[4:42:46 PM] Mitch Sundt: simplicity.
[4:43:34 PM] Mitch Sundt: If you want to describe what your form changes are, you can put it in the <descriptionText> of the FormListAPI.
[4:43:56 PM] Mitch Sundt: It doesn't belong in the submission.
[4:45:50 PM] Mitch Sundt: I need to get lunch -- signing out for now...
[4:53:37 PM] Jonathan Jackson: did we settle the formid / xmlns issue?
[4:55:41 PM] Jonathan Jackson: how about:  "version is a string representation of the version of the XForm.  It is strongly recommended that this is implemented as an integer-valued string that increments with changes to the XForm. The only requirement is that for any two forms of the same id and version, they are identical."
[4:56:38 PM] Drew Roos: that would be a SHOULD in spec-speak?
[4:57:12 PM] Anton de Winter: I'm fine with making it a string as well with a recommendation for ints
[5:06:57 PM] Anton de Winter: This is the current text for xlmns/id:
[5:06:58 PM] Anton de Winter: Either id or xmlns identifying the form id. These values should be in the form of scheme:value. For the id, the implementor's registered domain name should be used as part of the scheme (e.g., opendatakit.org:widgetForm). If specified, the id value takes precedence over any explicit xmlns declaration.
[5:09:30 PM] Jonathan Jackson: that's more confusing than saying nothing
[5:11:08 PM] Anton de Winter: wanna take a crack at re-wording?
[5:11:15 PM] Anton de Winter: I just copied and pasted...
[5:14:06 PM] Jonathan Jackson: it should be one or the other.
[5:14:09 PM] Jonathan Jackson: that was my question
[5:20:56 PM] Drew Roos: yaw's whole argument for id="" was that people don't want to deal with xmlns urls
[5:21:10 PM] Drew Roos: if the id requires a domain name, it's not appreciably different from an xmlns
[5:29:04 PM] Yaw Anokwa: Either id or xmlns identifying the form id. If xmlns, it should be a fully qualified url. If id, it should be an alphanumeric string. If specified, the id value takes precedence over any explicit xmlns declaration.
[5:30:05 PM] Matt Adams: Makes sense to me.
[5:31:48 PM] Matt Adams: I think that positions on several of these topics have been well covered. Maybe tomorrow is the day to summarize the contentious issues and put them to a vote?
[5:36:35 PM] Anton de Winter: Unfort tomorrow is not a weekdday
[5:36:47 PM] Anton de Winter: I'm happy with leaving the xmlns/id part as is in the spec
[5:36:50 PM] Matt Adams: Oh, right ;)
[5:36:53 PM] Anton de Winter: since it's been heavily discussed before
[5:37:08 PM] Anton de Winter: and it's not unreasonable..
[5:37:33 PM | Edited 5:37:43 PM] Clayton Sims: Wait, happy with leaving what, having two form ids?
[5:37:47 PM] Clayton Sims: the xmlns and the formid?
[5:39:05 PM | Edited 5:39:50 PM] Yaw Anokwa: dimagi doesn't work on weekends? weak sauce.
[5:45:23 PM] Jonathan Jackson: Yaw - are you proposing an either or?
[5:45:27 PM] Jonathan Jackson: with a trump?
[5:45:44 PM] Anton de Winter: Clayton, you have options for either or
[5:45:50 PM] Anton de Winter: with a precedent for one if there are both
[5:45:58 PM] Anton de Winter: formID is the more human readable of the two if that's what you're into
[5:46:09 PM] Anton de Winter: xmlns is for how we roll, with a full URI inlcuding a UUID
[5:46:12 PM] Jonathan Jackson: that's literally worse than no spec
[5:46:22 PM] Anton de Winter: that doesn't make sense.
[5:46:40 PM | Edited 5:46:49 PM] Jonathan Jackson: either, this, or that, if that is there, its used over this.
[5:47:10 PM] Clayton Sims: ...
[5:47:12 PM] Jonathan Jackson: so if there is an xmlns
[5:47:14 PM] Clayton Sims: == jon
[5:47:16 PM] Jonathan Jackson: and I edit an xform by hand
[5:47:22 PM] Jonathan Jackson: and I add a formid
[5:47:28 PM] Clayton Sims: That's basically saying "implement both standards fully because there's no guarantee anyone is doing either"
[5:47:29 PM] Jonathan Jackson: what is the expected behavior of anything receiving that xform?
[5:48:14 PM] Drew Roos: the id attr is for people who can't be bothered to do an xmlns
[5:48:23 PM] Drew Roos: xmlns should take precedence, if present
[5:48:37 PM] Clayton Sims: I really don't see the point to deciding to use two things to mean the same thing
[5:48:39 PM] Jonathan Jackson: you can't have precedence
[5:48:41 PM] Clayton Sims: and forcing people to use both
[5:48:50 PM] Jonathan Jackson: if you ahve precedence, you don't have a standard
[5:49:00 PM] Jonathan Jackson: because dimagi will do it one way, odk another, and then the forms aren't actually compatible.
[5:49:01 PM] Drew Roos: that's not true at all
[5:49:05 PM] Drew Roos: HTTP has tons of optional stuff
[5:49:13 PM] Jonathan Jackson: optional, yes.
[5:49:20 PM] Jonathan Jackson: but not trumping, to my knowledge.
[5:49:41 PM] Jonathan Jackson: we can make both optional
[5:49:43 PM] Jonathan Jackson: and neither required
[5:49:45 PM] Jonathan Jackson: (no spec)
[5:49:48 PM] Drew Roos: ok, well say an instance with both an xmlns and an id attr is non-conformant
[5:49:50 PM] Jonathan Jackson: and that woudl be better
[5:50:01 PM] Clayton Sims: ...
[5:50:08 PM] Clayton Sims: Why are we not just choosing one
[5:50:11 PM] Clayton Sims: this is not that hard
[5:50:12 PM] Jonathan Jackson: agreed.
[5:50:15 PM] Yaw Anokwa: id it is.
[5:50:27 PM] Jonathan Jackson: isnt' xmlns the "right" way to do xml?
[5:50:31 PM] Drew Roos: because xml validators will complain if your xmlns doesn't look like a url
[5:50:42 PM] Clayton Sims: that's mean of them
[5:50:45 PM] Drew Roos: that's the only reason for id to exist
[5:51:08 PM] Drew Roos: and, if i understand correctly, the only reason ODK is using it
[5:52:01 PM] Yaw Anokwa: i think we started using it before we knew that xmlns was suppose to be the unique id for an xml document.
[5:52:25 PM] Yaw Anokwa: and once we understood that, we adopted it. but. i think id is way way way clearer for human beings.
[5:52:52 PM] Yaw Anokwa: and i think most people don't use xmlns for anything like an id. they use it for a url.
[5:53:56 PM] Drew Roos: these people should read the xforms spec
[5:54:41 PM] Yaw Anokwa: you and i have different expectations of the people who design forms.
[5:55:09 PM] Drew Roos: not really
[5:55:25 PM] Drew Roos: but someone who can't understand something like xmlns probably shouldn't be messing with raw xml in the first place
[5:55:39 PM] Drew Roos: i.e., using a form designer
[5:56:24 PM] Anton de Winter: Agree with the sentiment that non techys should be using a form designer, so difficulty understanding xmlns (for humans) is a non issue
[5:56:24 PM] Mitch Sundt: xmlns is more often used to specify the grammar of a document, not its identity.  Look at server configuration files.
[5:56:43 PM] Mitch Sundt: Lots of xmlns assignments, and you get to choose what your default namespace is, by leaving off the :blah.
[5:56:50 PM | Edited 5:57:21 PM] Drew Roos: mitch, those are the same thing
[5:57:03 PM] Drew Roos: an instance template is a grammar for an xml document
[5:57:04 PM] Mitch Sundt: It doesn't actually dictate the identity of anything; just the grammar for that fragment fo the xml
[5:58:03 PM] Yaw Anokwa: non-techies edit raw forms all the time (especially since all the form designers are deficient in some way). and you want to make it obvious what things are.
[5:58:28 PM] Drew Roos: but saying 'xmlns is confusing' is like saying '<bind>s are confusing; let's change it'
[5:58:43 PM] Yaw Anokwa: yes. i think binds are stupid too.
[5:59:04 PM] Yaw Anokwa: but that's more core to xforms. xmlns isnt.
[5:59:05 PM] Drew Roos: but at a certain point it's not xforms anymore
[5:59:08 PM] Anton de Winter: Changing it is mission for 2.0
[5:59:11 PM] Drew Roos: it is core
[5:59:25 PM] Drew Roos: xmlns of the instance defines the type of data this xform produces
[5:59:42 PM] Yaw Anokwa: the rest of the world doesn't use xmlns to uniquely identify a form. we are.
[5:59:46 PM] Yaw Anokwa: that seems wrong.
[5:59:54 PM] Drew Roos: otherwise, all your custom instance nodes are part of the xforms namespace
[6:00:04 PM] Drew Roos: which, if we cared, would cause the form to not validate as an xform
[6:00:23 PM] Drew Roos: it's uniquely identifying the structure of the data this form produces!
[6:00:28 PM] Drew Roos: that's what xmlns _means_
[6:00:31 PM] Jonathan Jackson: If xmlns is the "right" way to do this, I think we should go with that.
[6:00:41 PM] Jonathan Jackson: even if its not the human easiest.
[6:00:58 PM] Jonathan Jackson: whereas version has no "right" answer,
[6:01:07 PM] Jonathan Jackson: this seems like it has  "right" answer
[6:03:41 PM] Mitch Sundt: I don't think specify that the default xmlns is the identity of the form is the "right" answer.  The following should be valid XML:
[6:03:41 PM] Mitch Sundt: <instance>
    <foo:geotagger xmlns:foo="geo_tagger_v2" >
     <foo:DeviceId/>
     <foo:Image/>
     <foo:Location/>
     <foo:Description/>
    </foo:geotagger>
   </instance>
[6:04:07 PM] Mitch Sundt: So if you have a tool that generates namespaced, named values, how do you know what the identity fo the form is?
[6:04:28 PM] Mitch Sundt: Note that the meta data fields were to use orx.
[6:04:48 PM] Drew Roos: but the top level xmlns for the resulting instance is the identity of the form
[6:05:42 PM] Drew Roos: and the system breaks down if you have multiple forms generating the same type of instance, such as a standardized instance in this case
[6:06:02 PM] Mitch Sundt: which would happen if you have versions, etc.
[6:06:16 PM] Mitch Sundt: and the user started adding questions
[6:06:29 PM] Drew Roos: isn't that why we have a version attribute?
[6:06:51 PM] Mitch Sundt: version + form Id define the structure.
[6:07:11 PM] Mitch Sundt: xmlns, if it were just formId would not be sufficent to characterize the submission
[6:07:23 PM] Drew Roos: why not?
[6:08:19 PM] Drew Roos: i guess i can concede the theoretical need to keep track of multiple xforms (note: not just multiple versions of the same form) that generate the same resulting instance
[6:08:31 PM] Drew Roos: but i'm not sure this has ever come up in the whole of our xforms experience
[6:08:37 PM] Mitch Sundt: If xmlns is dictating the structure of the xml, in the first form, without a "foo" question and with a "bar" question, it would have a different valid DTD than in the second.
[6:09:00 PM] Drew Roos: and if you want to do that, the "id" attribute should probably not be on the instance, but in the <model> tag
[6:09:31 PM] Mitch Sundt: the model tag doesn't get submitted up with the submission xml
[6:11:19 PM] Mitch Sundt: I would argue that the human-graspable concept is ( formId + version ) the mapping to xmlns is more for computer verification of the structure of the xml and should be kept separate.
[6:19:08 PM] Anton de Winter: It feels like we're diverging pretty hard here
[6:19:30 PM] Anton de Winter: instead of converging on a solution
[6:22:34 PM] Mitch Sundt: I hold firm to using id and version.  If id is not present, fall back to using xmlns of the tag.  If version is not present, it is handled as NULL
[6:22:53 PM] Yaw Anokwa: +1
[6:23:17 PM] Matt Adams: +1 that's what we're doing with those values (id/xmlns).  Seemed to make sense to me.
[6:23:57 PM] Anton de Winter: and hey that's what the spec says..
[6:23:59 PM] Anton de Winter: (mostly)
[6:29:52 PM] Anton de Winter: So, assuming that's settled, and versioning is settled
[6:29:59 PM] Anton de Winter: are we good for throwing this to an official vote?
[6:30:16 PM] Jonathan Jackson: just to be clear.
[6:30:22 PM] Jonathan Jackson: we are basically saying its not worth really solving this.
[6:30:28 PM] Jonathan Jackson: if tha'ts what we are saying, i'm fine with that.
[6:30:41 PM] Matt Adams: What happened with the one bit that I was concerned about - length of IDs?
[6:31:31 PM] Anton de Winter: What do you think would be a reasonable length limit?
[6:32:19 PM] Matt Adams: If we need a hard limit I think that 256 is going to cover the vast majority of uses.
[6:32:34 PM] Anton de Winter: cool, done.
[6:34:29 PM] Mitch Sundt: 80 characters is a reasonable length.  Nobody has URLs that are 256 characters.  Why would we allow anything that long?
[6:34:58 PM] Clayton Sims: 0, 1, or infinity
[6:36:01 PM] Anton de Winter: 80 seems a little short
[6:36:54 PM] Mitch Sundt: really??? This is not the Form Name, or the Description.
[6:37:05 PM] Drew Roos: omg who cares
[6:37:21 PM] Clayton Sims: We don't want people who are automatically dumping ID's from a foreign system to have to compress them to an arbitrary number
[6:37:28 PM] Drew Roos: does any spec have limits on how long attributes can be?
[6:37:38 PM] Clayton Sims: (this is a real, meaningful constraint for plenty of people we integrate with)
[6:37:44 PM] Mitch Sundt: guys. Databases care.
[6:38:01 PM] Anton de Winter: make it the limit of the db then.
[6:38:03 PM] Clayton Sims: If you're confident you won't ever need more than 80 characters you should be capable of hashing incoming strings into 80 chars
[6:38:04 PM] Anton de Winter: mysql limit
[6:38:59 PM] Mitch Sundt: mysql limit applies to the entire row, so it depends upon what else you're storing in the row.
[6:39:59 PM] Drew Roos: that sounds like mysql's problem
[6:40:08 PM] Anton de Winter: yep.
[6:40:10 PM] Mitch Sundt: Interoperability problem.
[6:40:58 PM] Anton de Winter: It might be that you guys need to make allowances for when your tech can't deal with a longer ID
[6:41:00 PM] Mitch Sundt: Again, do you have integration partners that currently have REEALLY long form ids?
[6:41:01 PM] Clayton Sims: Yeah, I don't think we should be arbitrarily restricting our spec to interpoerate with mysql
[6:41:03 PM] Anton de Winter: instead of making it part of the spec
[6:41:05 PM] Clayton Sims: mitch: Yes
[6:41:13 PM] Clayton Sims: They dump their table/internal names to stuff
[6:41:15 PM] Clayton Sims: which are long
[6:41:35 PM] Mitch Sundt: If you have a current requirement that 80 is too small for, we can increase it.  v2 is always easier to grow rather than saddle ourselves with an overly-generous value in v1
[6:41:35 PM] Clayton Sims: Just hash strings
[6:41:53 PM] Clayton Sims: an 80 character limit on an id is, like, something you would write in 1993
[6:41:58 PM] Mitch Sundt: hashes work for me.
[6:42:01 PM] Clayton Sims: "who would need a file name over 75 characters?"
[6:42:31 PM] Mitch Sundt: these days, I'd just do a sha-256 hash of the long string and call it good
[6:42:56 PM] Anton de Winter: I honestly think if you guys set an internal limit of 80 chars on aggregate, it will work fine in most cases. A length doesn't need to be part of the spec (esp versin 1.0 of the spec)
[6:43:03 PM] Clayton Sims: Cool, either way, the id on our side should be unbounded
[6:43:10 PM] Anton de Winter: if you run into problems, actively, it's something we can tackle for version 1.1
[6:43:14 PM] Anton de Winter: since we'll have hard data
[6:43:33 PM | Edited 6:43:36 PM] Anton de Winter: ("we drop 10% of submitted forms because their id's are too long, that's not ok")
[6:43:55 PM] Anton de Winter: but as of right now, it seems like a lot of speculation on something that doesn't need to be specified right now
[6:44:22 PM] Mitch Sundt: depends whether you want this spec to define the core interoperability or a loose, could-bite-you interoperability like the SQL spec.
[6:44:37 PM] Mitch Sundt: I would prefer more explicit interoperability
[6:44:46 PM] Anton de Winter: I think we should aim for looser and-at-least-get-it-done for 1.0
[6:44:49 PM] Anton de Winter: then tighten it up as needed
[6:45:00 PM] Mitch Sundt: tightening never happens
[6:45:35 PM] Anton de Winter: There are also ways for you to get around length limitations imposed by mysql
[6:47:08 PM] Mitch Sundt: Yes and no.
[6:47:42 PM] Drew Roos: including not accepting forms with super-long ids
[6:47:52 PM] Mitch Sundt: My argument is set a common lower limit (e.g., 80) if your server handles larger IDs, it is more capable, and the market will drive others to increase their sizes to meet the need.
[6:48:11 PM] Mitch Sundt: If you start large, there is no market pressure to change.
[6:48:30 PM] Mitch Sundt: The issue is if you create a form, can it be uploaded to any server, and used.
[6:49:21 PM] Drew Roos: are we going to limit how many nodes you can have in your instance?
[6:49:27 PM] Mitch Sundt: This is why I think we should have the version spec'd as an integer and the form id value limited to 80 characters.  Start small, if you have a form designer that does  something different, and you walk away from compatibility downstream, so be it.
[6:49:43 PM] Mitch Sundt: @Drew: no.
[6:50:17 PM] Mitch Sundt: The form designer, though, should be able to specify an integer if you run it in compatibility mode.
[6:50:59 PM] Mitch Sundt: Again, it isn't a limiting of the capability of the servers.  it is about establishing a common agreed upon minimal functionality.
[6:51:41 PM] Mitch Sundt: Add features and go beyond the standard, but still provide at least the common standard.
[6:52:01 PM] Anton de Winter: setting it low at 80 is basically asking us to immediately break from the standard
[6:52:13 PM] Anton de Winter: which makes the exercise somewhat pointless
[6:52:14 PM] Drew Roos: i don't understand how the mysql row constraint doesn't affect nearly every aspect of your xform handling
[6:52:21 PM] Drew Roos: and why similar limits aren't needed there
[6:52:21 PM] Mitch Sundt: what value do you need right now?
[6:53:26 PM] Anton de Winter: I don't think setting a limit makes much sense
[6:53:32 PM] Anton de Winter: Drew's question is pretty valid
[6:53:39 PM] Anton de Winter: are you going to limit the number of nodes in an instance?
[6:53:48 PM] Anton de Winter: because if we limit here, we should limit there
[6:53:54 PM] Anton de Winter: and while we're at it, limit the length of each nodeid
[6:54:02 PM] Anton de Winter: so that it all fits nicely in mysql
[6:54:36 PM] Mitch Sundt: If you need a longer string length for form ids, how about 249?  That is the maximum length for a searchable string in Google BigTable.  Anything beyond that would be onerous to support.
[6:54:58 PM] Mitch Sundt: As for the representation of the form, I have a loop that looks like this:
[6:55:03 PM] Mitch Sundt: for (;;) {
[6:55:20 PM] Mitch Sundt:  try { create table with columns
[6:55:26 PM] Mitch Sundt:  } catch ( failure ) {
[6:55:49 PM] Mitch Sundt:    divide table into tables with linked primary keys.
[6:55:51 PM] Mitch Sundt: }
[6:55:53 PM] Mitch Sundt: }
[6:56:20 PM] Mitch Sundt: i.e., I keep trying to create the tables, splitting them until the database stops complaining about not being able to create them.
[6:56:22 PM] Cory Zue: if you're already doing that why are you so concerned about these arbitrary limits?
[6:56:38 PM] Drew Roos: i'd be happy with a practical limit in the neighbrhood of 256 but still don't think a limit needs to be explicitly specified
[6:57:03 PM] Mitch Sundt: The tables have multiple columns.  Spliltting them moves the storage requirement to a different table.
[6:57:29 PM] Mitch Sundt: If you have one large field, the databases consume space storing that field.
[6:58:05 PM] Mitch Sundt: If you say 1000 characters, and my system only works for 249, I've got a problem supporting the standard.
[6:58:12 PM] Cory Zue: i think if we separate MUST and SHOULD then we can sandbox all the space/mysql constraints into SHOULDs and stop fighting ?
[6:59:05 PM] Mitch Sundt: that would mean the systems implementing the MUSTs are not necessarily interoperable.  I thought that was the goal.
[6:59:21 PM] Mitch Sundt: MUST support form ids of 249 characters or less.
[6:59:37 PM] Mitch Sundt: MUST support integer version values.
[6:59:48 PM] Drew Roos: is there a limit on how long the User-Agent header in HTTP can be?
[6:59:48 PM] Mitch Sundt: SHOULD support non-integer version values.
[7:00:33 PM] Cory Zue: drew, i don't think so
[7:00:47 PM] Cory Zue: mitch, i think that's fine?
[7:00:47 PM] Drew Roos: right
[7:01:22 PM] Drew Roos: if you get data that is larger than what you determine to be "reasonable", return an error
[7:02:12 PM] Mitch Sundt: as long as the MUST is what everyone knows will interoperate.
[7:02:20 PM] Mitch Sundt: i agree
[7:03:12 PM] Anton de Winter: ok
[7:03:29 PM] Anton de Winter: so MUST be 249 chars, SHOULD be any length
[7:04:00 PM] Anton de Winter: does that work?
[7:04:09 PM] Mitch Sundt: yes
[7:04:11 PM] Anton de Winter: sweet
[7:04:31 PM] Anton de Winter: The language you stated for version is something I'm also happy with including
[7:04:38 PM] Mitch Sundt: great
[7:04:53 PM] Anton de Winter: I'll likely kick off voting tomorrow but have it run into monday
[7:04:59 PM] Anton de Winter: or early tuesday
[7:05:08 PM] Mitch Sundt: ok.  I'll not be online over the weekend.
[7:05:17 PM] Anton de Winter: yeah, I'm not thinking that many people will be ;)
[7:05:38 PM] Mitch Sundt: You should search for uiversion in the spec -- I think there's some reference to it down at the bottom of the page that will need to be updated.
[7:05:40 PM | Edited 7:06:00 PM] Anton de Winter: but after all this discusison I think everyone has a good feel for how they want to vote (so it shouldn't take long)
[7:05:49 PM] Anton de Winter: cool, will do
[7:06:15 PM] Anton de Winter: Logs have also been archived https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaAPI
[7:06:21 PM] Anton de Winter: Chat logs*