XEmacs / CHANGES-ben-mule

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
List of changes in new Mule workspace:
--------------------------------------

Deleted files:

src/iso-wide.h
src/mule-charset.h
src/mule.c
src/ntheap.h
src/syscommctrl.h
lisp/files-nomule.el
lisp/help-nomule.el
lisp/mule/mule-help.el
lisp/mule/mule-init.el
lisp/mule/mule-misc.el
nt/config.h


Other deleted files, all zero-width and accidentally present:

src/events-mod.h
tests/Dnd/README.OffiX
tests/Dnd/dragtest.el
netinstall/README.xemacs
lib-src/srcdir-symlink.stamp

New files:

CHANGES-ben-mule
README.ben-mule-21-5
README.ben-separate-stderr
TODO.ben-mule-21-5
etc/TUTORIAL.{cs,es,nl,sk,sl}
etc/unicode/*
lib-src/make-mswin-unicode.pl
lisp/code-init.el
lisp/resize-minibuffer.el
lisp/unicode.el
lisp/mule/china-util.el
lisp/mule/cyril-util.el
lisp/mule/devan-util.el
lisp/mule/devanagari.el
lisp/mule/ethio-util.el
lisp/mule/indian.el
lisp/mule/japan-util.el
lisp/mule/korea-util.el
lisp/mule/lao-util.el
lisp/mule/lao.el
lisp/mule/mule-locale.txt
lisp/mule/mule-msw-init.el
lisp/mule/thai-util.el
lisp/mule/thai.el
lisp/mule/tibet-util.el
lisp/mule/tibetan.el
lisp/mule/viet-util.el
src/charset.h
src/intl-auto-encap-win32.c
src/intl-auto-encap-win32.h
src/intl-encap-win32.c
src/intl-win32.c
src/intl-x.c
src/mule-coding.c
src/text.c
src/text.h
src/unicode.c
src/s/win32-common.h
src/s/win32-native.h



gzip support:

-- new coding system `gzip' (bytes -> bytes); unfortunately, not quite
   working yet because it handles only the raw zlib format and not the
   higher-level gzip format (the zlib library is brain-damaged in that it
   provides low-level, stream-oriented API's only for raw zlib, and for
   gzip you have only high-level API's, which aren't useful for xemacs).
-- configure support (with-zlib).

configure changes:

- file-coding always compiled in.  eol detection is off by default on unix,
  non-mule, but can be enabled with configure option
  --with-default-eol-detection or command-line flag -eol.
- code that selects which files are compiled is mostly moved to
   Makefile.in.in.  see comment in Makefile.in.in.
- vestigial i18n3 code deleted.
- new cygwin mswin libs imm32 (input methods), mpr (user name enumeration).
- check for link, symlink.
- vfork-related code deleted.
- fix configure.usage. (delete --with-file-coding, --no-doc-file, add
  --with-default-eol-detection, --quick-build).
- nt/config.h has been eliminated and everything in it merged into
  config.h.in and s/windowsnt.h.  see config.h.in for more info.
- massive rewrite of s/windowsnt.h, m/windowsnt.h, s/cygwin32.h,
  s/mingw32.h.  common code moved into s/win32-common.h, s/win32-native.h.
- in nt/xemacs.mak,config.inc.samp, variable is called MULE, not HAVE_MULE,
  for consistency with sources.
- define TABDLY, TAB3 in freebsd.h (#### from where?)

Tutorial:

- massive rewrite; sync to FSF 21.0.106, switch focus to window systems,
  new sections on terminology and multiple frames, lots of fixes for
  current xemacs idioms.
- german version from Adrian mostly matching my changes.
- copy new tutorials from FSF (Spanish, Dutch, Slovak, Slovenian, Czech);
  not updated yet though.
- eliminate help-nomule.el and mule-help.el; merge into one single tutorial
  function, fix lots of problems, put back in help.el where it belongs.
  (there was some random junk in help-nomule -- string-width and make-char.
  string-width is now in subr.el with a single definition, and make-char in
  text.c.)

Sample init file:

- remove forward/backward buffer code, since it's now standard.
- when disabling C-x C-c, make it display a message saying how to exit, not
  just beep and complain "undefined".

Key bindings: (keymap.c, keydefs.el, help.el, etc.)

- M-home, M-end now move forward and backward in buffers; with Shift, stay
  within current group (e.g. all C files; same grouping as the gutter
  tabs). (bindings switch-to-{next/previous}-buffer[-in-group] in files.el)
  - needed to move code from gutter-items.el to buff-menu.el that's used by
    these bindings, since gutter-items.el is loaded only when the gutter is
    active and these bindings (and hence the code) is not (any more) gutter
    specific.
- new global vars global-tty-map and global-window-system-map specify key
  bindings for use only on TTY's or window systems, respectively.  this is
  used to make ESC ESC be keyboard-quit on window systems, but ESC ESC ESC
  on TTY's, where Meta + arrow keys may appear as ESC ESC O A or whatever.
  C-z on window systems is now zap-up-to-char, and iconify-frame is moved
  to C-Z.  ESC ESC is isearch-quit. (isearch-mode.el)
- document global-{tty,window-system}-map in various places; display them
  when you do C-h b.
- fix up function documentation in general for keyboard primitives.
  e.g. key-bindings now contains a detailed section on the steps prior to
  looking up in keymaps, i.e. function-key-map,
  keyboard-translate-table. etc.  define-key and other obvious starting
  points indicate where to look for more info.
- eliminate use and mention of grody advertised-undo and
  deprecated-help. (simple.el, startup.el, picture.el, menubar-items.el)

gnuclient, gnuserv:

- clean up headers a bit.
- use proper ms win idiom for checking for temp directory (TEMP or TMP, not
  TMPDIR).

throughout XEmacs sources:

- all #ifdef FILE_CODING statements removed from code.

I/O:

- use PATH_MAX consistently instead of MAXPATHLEN, MAX_PATH, etc.
- all code that does preprocessor games with C lib I/O functions (open,
  read) has been removed.  The code has been changed to call the correct
  function directly.  Functions that accept Intbyte * arguments for
  filenames and such and do automatic conversion to or from external format
  will be prefixed qxe...().  Functions that are retrying in case of EINTR
  are prefixed retry_...().  DONT_ENCAPSULATE is long-gone.
- never call getcwd() any more.  use our shadowed value always.

Strings:

- new qxe() string functions that accept Intbyte * as arguments.  These
  work exactly like the standard strcmp(), strcpy(), sprintf(), etc. except
  for the argument declaration differences.  We use these whenever we have
  Intbyte * strings, which is quite often.
- new fun build_intstring() takes an Intbyte *.  also new funs
  build_msg_intstring (like build_intstring()) and build_msg_string (like
  build_string()) to do a GETTEXT() before building the
  string. (elimination of old build_translated_string(), replaced by
  build_msg_string()).
- the doprnt.c external entry points have been completely rewritten to be
  more useful and have more sensible names.  We now have, for example,
  versions that work exactly like sprintf() but return a malloc()ed string.
- function intern_int() for Intbyte * arguments, like intern().
- numerous places throughout code where char * replaced with something
  else, e.g. Char_ASCII *, Intbyte *, Char_Binary *, etc.  same with
  unsigned char *, going to UChar_Binary *, etc.
- code in print.c that handles stdout, stderr rewritten.
- places that print to stderr directly replaced with stderr_out().
- new convenience functions write_fmt_string(), write_fmt_string_lisp(), stderr_out_lisp(), write_string().

Allocation, Objects, Lisp Interpreter:

- automatically use "managed lcrecord" code when allocating.  any lcrecord
  can be put on a free list with free_lcrecord().
- record_unwind_protect() returns the old spec depth.
- unbind_to() now takes only one arg.  use unbind_to_1() if you want the
  2-arg version, with GC protection of second arg.
- new funs to easily inhibit GC. ({begin,end}_gc_forbidden()) use them in
  places where gc is currently being inhibited in a more ugly fashion.
  also, we disable GC in certain strategic places where string data is
  often passed in, e.g. dfc functions, print functions.
- major improvements to eistring code, fleshing out of missing funs.
- make_buffer() -> wrap_buffer() for consistency with other objects; same
  for make_frame() -> wrap_frame() and make_console() -> wrap_console().
- better documentation in condition-case.
- new convenience funs record_unwind_protect_freeing() and
  record_unwind_protect_freeing_dynarr() for conveniently setting up an
  unwind-protect to xfree() or Dynarr_free() a pointer.

Init code:

- lots of init code rewritten to be mule-correct.

Processes:

- always call egetenv(), never getenv(), for mule correctness.

s/m files:

- removal of unused DATA_END, TEXT_END, SYSTEM_PURESIZE_EXTRA, HAVE_ALLOCA
  (automatically determined)
- removal of vfork references (we no longer use vfork)


make-docfile:

- clean up headers a bit.
- allow .obj to mean equivalent .c, just like for .o.
- allow specification of a "response file" (a command-line argument
  beginning with @, specifying a file containing further command-line
  arguments) -- a standard mswin idiom to avoid potential command-line
  limits and to simplify makefiles.  use this in xemacs.mak.

debug support:

- (cmdloop.el) new var breakpoint-on-error, which breaks into the C
  debugger when an unhandled error occurs noninteractively.  useful when
  debugging errors coming out of complicated make scripts, e.g. package
  compilation, since you can set this through an env var.
- (startup.el) new env var XEMACSDEBUG, specifying a Lisp form executed
  early in the startup process; meant to be used for turning on debug flags
  such as breakpoint-on-error or stack-trace-on-error, to track down
  noninteractive errors.
- (cmdloop.el) removed non-working code in command-error to display a
  backtrace on debug-on-error.  use stack-trace-on-error instead to get
  this.
- (process.c) new var debug-process-io displays data sent to and received
  from a process.
- (alloc.c) staticpros have name stored with them for easier debugging.
- (emacs.c) code that handles fatal errors consolidated and rewritten.
  much more robust and correctly handles all fatal exits on mswin
  (e.g. aborts, not previously handled right).

command line (startup.el, emacs.c):

- new option -eol to enable auto EOL detection under non-mule unix.
- new option -nuni (--no-unicode-lib-calls) to force use of non-Unicode
  API's under Windows NT, mostly for debugging purposes.
- help message fixed up (divided into sections), existing problem causing
  incomplete output fixed, undocumented options documented.

startup.el:

- move init routines from before-init-hook or after-init-hook; just call
  them directly (init-menubar-at-startup, init-mule-at-startup).

frame.el:

- delete old commented-out code.

Mule changes:

Major:

- the code that handles the details of processing multilingual text has
  been consolidated to make it easier to extend it.  it has been yanked out
  of various files (buffer.h, mule-charset.h, lisp.h, insdel.c, fns.c,
  file-coding.c, etc.) and put into text.c and text.h.  mule-charset.h has
  also been renamed charset.h.  all long comments concerning the
  representations and their processing have been consolidated into text.c.
- major rewriting of file-coding.  it's mostly abstracted into coding
  systems that are defined by methods (similar to devices and
  specifiers), with the ultimate aim being to allow non-i18n coding
  systems such as gzip.  there is a "chain" coding system that allows
  multiple coding systems to be chained together. (it doesn't yet
  have the concept that either end of a coding system can be bytes or
  chars; this needs to be added.)
- large amounts of code throughout the code base have been Mule-ized,
  not just Windows code.
- total rewriting of OS locale code.  it notices your locale at startup and
  sets the language environment accordingly, and calls setlocale() and sets
  LANG when you change the language environment.  new language environment
  properties locale, mswindows-locale, cygwin-locale, native-coding-system,
  to determine langenv from locale and vice-versa; fix all language
  environments (lots of language files).  langenv startup code rewritten.
  many new functions to convert between locales, language environments,
  etc.
- major overhaul of the way default values for the various coding system
  variables are handled.  all default values are collected into one
  location, a new file code-init.el, which provides a unified mechanism for
  setting and querying what i call "basic coding system variables" (which
  may be aliases, parts of conses, etc.) and a mechanism of different
  configurations (Windows w/Mule, Windows w/o Mule, Unix w/Mule, Unix w/o
  Mule, unix w/o Mule but w/auto EOL), each of which specifies a set of
  default values.  we determine the configuration at startup and set all
  the values in one place. (code-init.el, code-files.el, coding.el, ...)
- i copied the remaining language-specific files from fsf.  i made
  some minor changes in certain cases but for the most part the stuff
  was just copied and may not work.
- ms windows mule support, with full unicode support.  required font,
  redisplay, event, other changes.  ime support from ikeyama.

User-Visible Changes:

Lisp-Visible Changes:

- ensure that `escape-quoted' works correctly even without Mule support and
  use it for all auto-saves. (auto-save.el, fileio.c, coding.el, files.el)
- new var buffer-file-coding-system-when-loaded specifies the actual coding
  system used when the file was loaded (buffer-file-coding-system is
  usually the same, but may be changed because it controls how the file is
  written out).  use it in revert-buffer (files.el, code-files.el) and in
  new submenu File->Revert Buffer with Specified Encoding
  (menubar-items.el).
- improve docs on how the coding system is determined when a file is read
  in; improved docs are in both find-file and insert-file-contents and a
  reference to where to find them is in
  buffer-file-coding-system-for-read. (files.el, code-files.el)
- new (brain-damaged) FSF way of calling post-read-conversion (only one
  arg, not two) is supported, along with our two-argument way, as best we
  can. (code-files.el)
- add inexplicably missing var default-process-coding-system.  use it.  get
  rid of former hacked-up way of setting these defaults using
  comint-exec-hook.  also fun
  set-buffer-process-coding-system. (code-process.el, code-cmds.el, process.c)
- remove function set-default-coding-systems; replace with
  set-default-output-coding-systems, which affects only the output defaults
  (buffer-file-coding-system, output half of
  default-process-coding-system).  the input defaults should not be set by
  this because they should always remain `undecided' in normal
  circumstances.  fix prefer-coding-system to use the new function and
  correct its docs.
- fix bug in coding-system-change-eol-conversion (code-cmds.el)
- recognize all eol types in prefer-coding-system (code-cmds.el)
- rewrite coding-system-category to be correct (coding.el)

Internal Changes:

- Separate encoding and decoding lstreams have been combined into a single
  coding lstream.  Functions make_encoding_*_stream and
  make_decoding_*_stream have been combined into make_coding_*_stream,
  which takes an argument specifying whether encode or decode is wanted.
- remove last vestiges of I18N3, I18N4 code.
- ascii optimization for strings: we keep track of the number of ascii
  chars at the beginning and use this to optimize byte<->char conversion on
  strings.
- mule-misc.el, mule-init.el deleted; code in there either deleted,
  rewritten, or moved to another file.
- mule.c deleted.
- move non-Mule-specific code out of mule-cmds.el into code-cmds.el. (coding-system-change-text-conversion; remove duplicate coding-system-change-eol-conversion)
- remove duplicate set-buffer-process-coding-system (code-cmds.el)
- add some commented-out code from FSF mule-cmds.el
  (find-coding-systems-region-subset-p, find-coding-systems-region,
  find-coding-systems-string, find-coding-systems-for-charsets,
  find-multibyte-characters, last-coding-system-specified,
  select-safe-coding-system, select-message-coding-system) (code-cmds.el)
- remove obsolete alias pathname-coding-system, function set-pathname-coding-system (coding.el)
- remove coding-system property doc-string; split into `description'
  (short, for menu items) and `documentation' (long); correct coding system
  defns (coding.el, file-coding.c, lots of language files)
- move coding-system-base into C and make use of internal info (coding.el, file-coding.c)
- move undecided defn into C (coding.el, file-coding.c)
- use define-coding-system-alias, not copy-coding-system (coding.el)
- new coding system iso-8859-6 for arabic
- delete windows-1251 support from cyrillic.el; we do it automatically
- remove setup-*-environment as per FSF 21
- rewrite european.el with lang envs for each language, so we can specify the locale
- fix corruption in greek.el
- sync japanese.el with FSF 20.6
- fix warnings in mule-ccl.el
- move FSF compat Mule fns from obsolete.el to mule-charset.el
- eliminate unused truncate-string{-to-width}
- make-coding-system accepts (but ignores) the additional properties
  present in the fsf version, for compatibility.
- i fixed the iso2022 handling so it will correctly read in files
  containing unknown charsets, creating a "temporary" charset which
  can later be overwritten by the real charset when it's defined.
  this allows iso2022 elisp files with literals in strange languages
  to compile correctly under mule.  i also added a hack that will
  correctly read in and write out the emacs-specific "composition"
  escape sequences, i.e. ESC 0 through ESC 4.  this means that my
  workspace correctly compiles the new file devanagari.el that i added.
- elimination of string-to-char-list (use string-to-list)
- elimination of junky define-charset

Search:

- make regex routines reentrant, since they're sometimes called
  reentrantly. (see regex.c for a description of how.) all global variables
  used by the regex routines get pushed onto a stack by the callers before
  being set, and are restored when finished.  redo the preprocessor flags
  controlling REL_ALLOC in conjunction with this.

Selection:

- fix msw selection code for Mule.  proper encoding for
  RegisterClipboardFormat.  store selection as CF_UNICODETEXT, which will
  get converted to the other formats.  don't respond to destroy messages
  from EmptyClipboard().

Menubar:

- move menu-splitting code (menu-split-long-menu, etc.) from font-menu.el
  to menubar-items.el and redo its algorithm; use in various items with
  long generated menus; rename to remove `font-' from beginning of
  functions but keep old names as aliases
- new fn menu-sort-menu
- new items Open With Specified Encoding, Revert Buffer with Specified Encoding
- split Mule menu into Encoding (non-Mule-specific; includes new item to
  control EOL auto-detection) and International submenus on Options,
  International on Help
- redo items Grep All Files in Current Directory {and Below} using stuff
  from sample init.el
- Debug on Error and friends now affect current session only; not saved
- maybe-add-init-button -> init-menubar-at-startup and call explicitly from startup.el
- don't use charset-registry in msw-font-menu.el; it's only for X

Process:

- Move setenv from packages; synch setenv/getenv with 21.0.105

Unicode support:

- translation tables added in etc/unicode
- new files unicode.c, unicode.el containing unicode coding systems and
  support; old code ripped out of file-coding.c
- translation tables read in at startup (NEEDS WORK TO MAKE IT MORE EFFICIENT)
- support CF_TEXT, CF_UNICODETEXT in select.el
- encapsulation code added so that we can support both Windows 9x and NT in
  a single executable, determining at runtime whether to call the Unicode
  or non-Unicode API.  encapsulated routines in intl-encap-win32.c
  (non-auto-generated) and intl-auto-encap-win32.[ch] (auto-generated).
  code generator in lib-src/make-mswin-unicode.pl.  changes throughout the
  code to use the wide structures (W suffix) and call the encapsulated
  Win32 API routines (qxe prefix).  calling code needs to do proper
  conversion of text using new coding systems Qmswindows_tstr,
  Qmswindows_unicode, or Qmswindows_multibyte. (the first points to one of
  the other two.)


File-coding rewrite:

The coding system code has been majorly rewritten.  It's abstracted into
coding systems that are defined by methods (similar to devices and
specifiers).  The types of conversions have also been
generalized. Formerly, decoding always converted bytes to characters and
encoding the reverse (these are now called "text file converters"), but
conversion can now happen either to or from bytes or characters.  This
allows coding systems such as `gzip' and `base64' to be written.  When
specifying such a coding system to an operation that expects a text file
converter (such as reading in or writing out a file), the appropriate
coding systems to convert between bytes and characters are automatically
inserted into the conversion chain as necessary.  To facilitate creating
such chains, a special coding system called "chain" has been created, which
chains together two or more coding systems.

Encoding detection has also been abstracted.  Detectors are logically
separate from coding systems, and each detector defines one or more
categories. (For example, the detector for Unicode defines categories such
as UTF-8, UTF-16, UCS-4, and UTF-7.) When a particular detector is given a
piece of text to detect, it determines likeliness values (seven of them,
from 3 [most likely] to -3 [least likely]; specific criteria are defined
for each possible value).  All detectors are run in parallel on a
particular piece of text, and the results tabulated together to determine
the actual encoding of the text.

Encoding and decoding are now completely parallel operations, and the
former "encoding" and "decoding" lstreams have been combined into a single
"coding" lstream.  Coding system methods that were formerly split in such a
fashion have also been combined.
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.