Source

go / doc / articles / laws_of_reflection.html

Full commit
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
<!--{
	"Title": "The Laws of Reflection",
	"Template": true
}-->

<p>
Reflection in computing is the
ability of a program to examine its own structure, particularly
through types; it's a form of metaprogramming. It's also a great
source of confusion.
</p>

<p>
In this article we attempt to clarify things by explaining how
reflection works in Go. Each language's reflection model is
different (and many languages don't support it at all), but
this article is about Go, so for the rest of this article the word
"reflection" should be taken to mean "reflection in Go".
</p>

<p><b>Types and interfaces</b></p>

<p>
Because reflection builds on the type system, let's start with a
refresher about types in Go.
</p>

<p>
Go is statically typed. Every variable has a static type, that is,
exactly one type known and fixed at compile time: <code>int</code>,
<code>float32</code>, <code>*MyType</code>, <code>[]byte</code>,
and so on. If we declare
</p>

{{code "/doc/progs/interface.go" `/type MyInt/` `/STOP/`}}

<p>
then <code>i</code> has type <code>int</code> and <code>j</code>
has type <code>MyInt</code>. The variables <code>i</code> and
<code>j</code> have distinct static types and, although they have
the same underlying type, they cannot be assigned to one another
without a conversion.
</p>

<p>
One important category of type is interface types, which represent
fixed sets of methods. An interface variable can store any concrete
(non-interface) value as long as that value implements the
interface's methods. A well-known pair of examples is
<code>io.Reader</code> and <code>io.Writer</code>, the types
<code>Reader</code> and <code>Writer</code> from the <a href=
"http://golang.org/pkg/io/">io package</a>:
</p>

{{code "/doc/progs/interface.go" `/// Reader/` `/STOP/`}}

<p>
Any type that implements a <code>Read</code> (or
<code>Write</code>) method with this signature is said to implement
<code>io.Reader</code> (or <code>io.Writer</code>). For the
purposes of this discussion, that means that a variable of type
<code>io.Reader</code> can hold any value whose type has a
<code>Read</code> method:
</p>

{{code "/doc/progs/interface.go" `/func readers/` `/STOP/`}}

<p>
It's important to be clear that whatever concrete value
<code>r</code> may hold, <code>r</code>'s type is always
<code>io.Reader</code>: Go is statically typed and the static type
of <code>r</code> is <code>io.Reader</code>.</p>

<p>
An extremely important example of an interface type is the empty
interface:
</p>

<pre>
interface{}
</pre>

<p>
It represents the empty set of methods and is satisfied by any
value at all, since any value has zero or more methods.
</p>

<p>
Some people say that Go's interfaces are dynamically typed, but
that is misleading. They are statically typed: a variable of
interface type always has the same static type, and even though at
run time the value stored in the interface variable may change
type, that value will always satisfy the interface.
</p>

<p>
We need to be precise about all this because reflection and
interfaces are closely related.
</p>

<p><b>The representation of an interface</b></p>

<p>
Russ Cox has written a <a href=
"http://research.swtch.com/2009/12/go-data-structures-interfaces.html">
detailed blog post</a> about the representation of interface values
in Go. It's not necessary to repeat the full story here, but a
simplified summary is in order.
</p>

<p>
A variable of interface type stores a pair: the concrete value
assigned to the variable, and that value's type descriptor.
To be more precise, the value is the underlying concrete data item
that implements the interface and the type describes the full type
of that item. For instance, after
</p>

{{code "/doc/progs/interface.go" `/func typeAssertions/` `/STOP/`}}

<p>
<code>r</code> contains, schematically, the (value, type) pair,
(<code>tty</code>, <code>*os.File</code>). Notice that the type
<code>*os.File</code> implements methods other than
<code>Read</code>; even though the interface value provides access
only to the <code>Read</code> method, the value inside carries all
the type information about that value. That's why we can do things
like this:
</p>

{{code "/doc/progs/interface.go" `/var w io.Writer/` `/STOP/`}}

<p>
The expression in this assignment is a type assertion; what it
asserts is that the item inside <code>r</code> also implements
<code>io.Writer</code>, and so we can assign it to <code>w</code>.
After the assignment, <code>w</code> will contain the pair
(<code>tty</code>, <code>*os.File</code>). That's the same pair as
was held in <code>r</code>. The static type of the interface
determines what methods may be invoked with an interface variable,
even though the concrete value inside may have a larger set of
methods.
</p>

<p>
Continuing, we can do this:
</p>

{{code "/doc/progs/interface.go" `/var empty interface{}/` `/STOP/`}}

<p>
and our empty interface value <code>e</code> will again contain
that same pair, (<code>tty</code>, <code>*os.File</code>). That's
handy: an empty interface can hold any value and contains all the
information we could ever need about that value.
</p>

<p>
(We don't need a type assertion here because it's known statically
that <code>w</code> satisfies the empty interface. In the example
where we moved a value from a <code>Reader</code> to a
<code>Writer</code>, we needed to be explicit and use a type
assertion because <code>Writer</code>'s methods are not a
subset of <code>Reader</code>'s.)
</p>

<p>
One important detail is that the pair inside an interface always
has the form (value, concrete type) and cannot have the form
(value, interface type). Interfaces do not hold interface
values.
</p>

<p>
Now we're ready to reflect.
</p>

<p><b>The first law of reflection</b></p>

<p><b>1. Reflection goes from interface value to reflection object.</b></p>

<p>
At the basic level, reflection is just a mechanism to examine the
type and value pair stored inside an interface variable. To get
started, there are two types we need to know about in
<a href="http://golang.org/pkg/reflect">package reflect</a>:
<a href="http://golang.org/pkg/reflect/#Type">Type</a> and
<a href="http://golang.org/pkg/reflect/#Value">Value</a>. Those two types
give access to the contents of an interface variable, and two
simple functions, called <code>reflect.TypeOf</code> and
<code>reflect.ValueOf</code>, retrieve <code>reflect.Type</code>
and <code>reflect.Value</code> pieces out of an interface value.
(Also, from the <code>reflect.Value</code> it's easy to get
to the <code>reflect.Type</code>, but let's keep the
<code>Value</code> and <code>Type</code> concepts separate for
now.)
</p>

<p>
Let's start with <code>TypeOf</code>:
</p>

{{code "/doc/progs/interface2.go" `/package main/` `/STOP main/`}}

<p>
This program prints
</p>

<pre>
type: float64
</pre>

<p>
You might be wondering where the interface is here, since the
program looks like it's passing the <code>float64</code>
variable <code>x</code>, not an interface value, to
<code>reflect.TypeOf</code>. But it's there; as <a href=
"http://golang.org/pkg/reflect/#Type.TypeOf">godoc reports</a>, the
signature of <code>reflect.TypeOf</code> includes an empty
interface:
</p>

<pre>
// TypeOf returns the reflection Type of the value in the interface{}.
func TypeOf(i interface{}) Type
</pre>

<p>
When we call <code>reflect.TypeOf(x)</code>, <code>x</code> is
first stored in an empty interface, which is then passed as the
argument; <code>reflect.TypeOf</code> unpacks that empty interface
to recover the type information.
</p>

<p>
The <code>reflect.ValueOf</code> function, of course, recovers the
value (from here on we'll elide the boilerplate and focus just on
the executable code):
</p>

{{code "/doc/progs/interface2.go" `/START f9/` `/STOP/`}}

<p>
prints
</p>

<pre>
value: &lt;float64 Value&gt;
</pre>

<p>
Both <code>reflect.Type</code> and <code>reflect.Value</code> have
lots of methods to let us examine and manipulate them. One
important example is that <code>Value</code> has a
<code>Type</code> method that returns the <code>Type</code> of a
<code>reflect.Value</code>. Another is that both <code>Type</code>
and <code>Value</code> have a <code>Kind</code> method that returns
a constant indicating what sort of item is stored:
<code>Uint</code>, <code>Float64</code>, <code>Slice</code>, and so
on. Also methods on <code>Value</code> with names like
<code>Int</code> and <code>Float</code> let us grab values (as
<code>int64</code> and <code>float64</code>) stored inside:
</p>

{{code "/doc/progs/interface2.go" `/START f1/` `/STOP/`}}

<p>
prints
</p>

<pre>
type: float64
kind is float64: true
value: 3.4
</pre>

<p>
There are also methods like <code>SetInt</code> and
<code>SetFloat</code> but to use them we need to understand
settability, the subject of the third law of reflection, discussed
below.
</p>

<p>
The reflection library has a couple of properties worth singling
out. First, to keep the API simple, the "getter" and "setter"
methods of <code>Value</code> operate on the largest type that can
hold the value: <code>int64</code> for all the signed integers, for
instance. That is, the <code>Int</code> method of
<code>Value</code> returns an <code>int64</code> and the
<code>SetInt</code> value takes an <code>int64</code>; it may be
necessary to convert to the actual type involved:
</p>

{{code "/doc/progs/interface2.go" `/START f2/` `/STOP/`}}

<p>
The second property is that the <code>Kind</code> of a reflection
object describes the underlying type, not the static type. If a
reflection object contains a value of a user-defined integer type,
as in
</p>

{{code "/doc/progs/interface2.go" `/START f3/` `/STOP/`}}

<p>
the <code>Kind</code> of <code>v</code> is still
<code>reflect.Int</code>, even though the static type of
<code>x</code> is <code>MyInt</code>, not <code>int</code>. In
other words, the <code>Kind</code> cannot discriminate an int from
a <code>MyInt</code> even though the <code>Type</code> can.
</p>

<p><b>The second law of reflection</b></p>

<p><b>2. Reflection goes from reflection object to interface
value.</b></p>

<p>
Like physical reflection, reflection in Go generates its own
inverse.
</p>

<p>
Given a <code>reflect.Value</code> we can recover an interface
value using the <code>Interface</code> method; in effect the method
packs the type and value information back into an interface
representation and returns the result:
</p>

<pre>
// Interface returns v's value as an interface{}.
func (v Value) Interface() interface{}
</pre>

<p>
As a consequence we can say
</p>

{{code "/doc/progs/interface2.go" `/START f3b/` `/STOP/`}}

<p>
to print the <code>float64</code> value represented by the
reflection object <code>v</code>.
</p>

<p>
We can do even better, though. The arguments to
<code>fmt.Println</code>, <code>fmt.Printf</code> and so on are all
passed as empty interface values, which are then unpacked by the
<code>fmt</code> package internally just as we have been doing in
the previous examples. Therefore all it takes to print the contents
of a <code>reflect.Value</code> correctly is to pass the result of
the <code>Interface</code> method to the formatted print
routine:
</p>

{{code "/doc/progs/interface2.go" `/START f3c/` `/STOP/`}}

<p>
(Why not <code>fmt.Println(v)</code>? Because <code>v</code> is a
<code>reflect.Value</code>; we want the concrete value it holds.)
Since our value is a <code>float64</code>, we can even use a
floating-point format if we want:
</p>

{{code "/doc/progs/interface2.go" `/START f3d/` `/STOP/`}}

<p>
and get in this case
</p>

<pre>
3.4e+00
</pre>

<p>
Again, there's no need to type-assert the result of
<code>v.Interface()</code> to <code>float64</code>; the empty
interface value has the concrete value's type information inside
and <code>Printf</code> will recover it.
</p>

<p>
In short, the <code>Interface</code> method is the inverse of the
<code>ValueOf</code> function, except that its result is always of
static type <code>interface{}</code>.
</p>

<p>
Reiterating: Reflection goes from interface values to reflection
objects and back again.
</p>

<p><b>The third law of reflection</b></p>

<p><b>3. To modify a reflection object, the value must be settable.</b></p>

<p>
The third law is the most subtle and confusing, but it's easy
enough to understand if we start from first principles.
</p>

<p>
Here is some code that does not work, but is worth studying.
</p>

{{code "/doc/progs/interface2.go" `/START f4/` `/STOP/`}}

<p>
If you run this code, it will panic with the cryptic message
</p>

<pre>
panic: reflect.Value.SetFloat using unaddressable value
</pre>

<p>
The problem is not that the value <code>7.1</code> is not
addressable; it's that <code>v</code> is not settable. Settability
is a property of a reflection <code>Value</code>, and not all
reflection <code>Values</code> have it.
</p>

<p>
The <code>CanSet</code> method of <code>Value</code> reports the
settability of a <code>Value</code>; in our case,
</p>

{{code "/doc/progs/interface2.go" `/START f5/` `/STOP/`}}

<p>
prints
</p>

<pre>
settability of v: false
</pre>

<p>
It is an error to call a <code>Set</code> method on an non-settable
<code>Value</code>. But what is settability?
</p>

<p>
Settability is a bit like addressability, but stricter. It's the
property that a reflection object can modify the actual storage
that was used to create the reflection object. Settability is
determined by whether the reflection object holds the original
item. When we say
</p>

{{code "/doc/progs/interface2.go" `/START f6/` `/STOP/`}}

<p>
we pass a <em>copy</em> of <code>x</code> to
<code>reflect.ValueOf</code>, so the interface value created as the
argument to <code>reflect.ValueOf</code> is a <em>copy</em> of
<code>x</code>, not <code>x</code> itself. Thus, if the
statement
</p>

{{code "/doc/progs/interface2.go" `/START f6b/` `/STOP/`}}

<p>
were allowed to succeed, it would not update <code>x</code>, even
though <code>v</code> looks like it was created from
<code>x</code>. Instead, it would update the copy of <code>x</code>
stored inside the reflection value and <code>x</code> itself would
be unaffected. That would be confusing and useless, so it is
illegal, and settability is the property used to avoid this
issue.
</p>

<p>
If this seems bizarre, it's not. It's actually a familiar situation
in unusual garb. Think of passing <code>x</code> to a
function:
</p>

<pre>
f(x)
</pre>

<p>
We would not expect <code>f</code> to be able to modify
<code>x</code> because we passed a copy of <code>x</code>'s value,
not <code>x</code> itself. If we want <code>f</code> to modify
<code>x</code> directly we must pass our function the address of
<code>x</code> (that is, a pointer to <code>x</code>):</p>

<p>
<code>f(&amp;x)</code>
</p>

<p>
This is straightforward and familiar, and reflection works the same
way. If we want to modify <code>x</code> by reflection, we must
give the reflection library a pointer to the value we want to
modify.
</p>

<p>
Let's do that. First we initialize <code>x</code> as usual
and then create a reflection value that points to it, called
<code>p</code>.
</p>

{{code "/doc/progs/interface2.go" `/START f7/` `/STOP/`}}

<p>
The output so far is
</p>

<pre>
type of p: *float64
settability of p: false
</pre>

<p>
The reflection object <code>p</code> isn't settable, but it's not
<code>p</code> we want to set, it's (in effect) <code>*p</code>. To
get to what <code>p</code> points to, we call the <code>Elem</code>
method of <code>Value</code>, which indirects through the pointer,
and save the result in a reflection <code>Value</code> called
<code>v</code>:
</p>

{{code "/doc/progs/interface2.go" `/START f7b/` `/STOP/`}}

<p>
Now <code>v</code> is a settable reflection object, as the output
demonstrates,
</p>

<pre>
settability of v: true
</pre>

<p>
and since it represents <code>x</code>, we are finally able to use
<code>v.SetFloat</code> to modify the value of
<code>x</code>:
</p>

{{code "/doc/progs/interface2.go" `/START f7c/` `/STOP/`}}

<p>
The output, as expected, is
</p>

<pre>
7.1
7.1
</pre>

<p>
Reflection can be hard to understand but it's doing exactly what
the language does, albeit through reflection <code>Types</code> and
<code>Values</code> that can disguise what's going on. Just keep in
mind that reflection Values need the address of something in order
to modify what they represent.
</p>

<p><b>Structs</b></p>

<p>
In our previous example <code>v</code> wasn't a pointer itself, it
was just derived from one. A common way for this situation to arise
is when using reflection to modify the fields of a structure. As
long as we have the address of the structure, we can modify its
fields.
</p>

<p>
Here's a simple example that analyzes a struct value,
<code>t</code>. We create the reflection object with the address of
the struct because we'll want to modify it later. Then we set
<code>typeOfT</code> to its type and iterate over the fields using
straightforward method calls (see 
<a href="http://golang.org/pkg/reflect/">package reflect</a> for details).
Note that we extract the names of the fields from the struct type,
but the fields themselves are regular <code>reflect.Value</code>
objects.
</p>

{{code "/doc/progs/interface2.go" `/START f8/` `/STOP/`}}

<p>
The output of this program is
</p>

<pre>
0: A int = 23
1: B string = skidoo
</pre>

<p>
There's one more point about settability introduced in
passing here: the field names of <code>T</code> are upper case
(exported) because only exported fields of a struct are
settable.
</p>

<p>
Because <code>s</code> contains a settable reflection object, we
can modify the fields of the structure.
</p>

{{code "/doc/progs/interface2.go" `/START f8b/` `/STOP/`}}

<p>
And here's the result:
</p>

<pre>
t is now {77 Sunset Strip}
</pre>

<p>
If we modified the program so that <code>s</code> was created from
<code>t</code>, not <code>&amp;t</code>, the calls to
<code>SetInt</code> and <code>SetString</code> would fail as the
fields of <code>t</code> would not be settable.
</p>

<p><b>Conclusion</b></p>

<p>
Here again are the laws of reflection:
</p>

<ol>
<li>Reflection goes from interface value to reflection
object.</li>
<li>Reflection goes from reflection object to interface
value.</li>
<li>To modify a reflection object, the value must be settable.</li>
</ol>

<p>
Once you understand these laws reflection in Go becomes much easier
to use, although it remains subtle. It's a powerful tool that
should be used with care and avoided unless strictly
necessary.
</p>

<p>
There's plenty more to reflection that we haven't covered &mdash;
sending and receiving on channels, allocating memory, using slices
and maps, calling methods and functions &mdash; but this post is
long enough. We'll cover some of those topics in a later
article.
</p>