Migrate streams library to go-fed.

streams/README.md

@@ -0,0 +1,199 @@
+# streams
+
+The `streams` package provides static types for Core and Extended types to the
+[ActivityStream Vocabulary](https://www.w3.org/TR/activitystreams-vocabulary).
+The library is battle-tested against the `vocabulary-ex*-jsonld.json`
+[test documents](https://github.com/w3c-social/activitystreams-test-documents)
+in addition to usual unit tests.
+
+Its mission is simple: Provide meaningful static types for the ActivityStream
+Vocabulary in golang. This library is a convenience layer on top of the
+`activity/vocab` library, which gives unfettered access to the data types.
+
+This library is entirely code-generated by the `activity/tools/streams/gen`
+library and `activity/tools/streams` tool. Run `go generate` to refresh the
+library, which requires `$GOPATH/bin` to be on your `$PATH`.
+
+**Consider using this library and falling back to `activity/vocab` only when
+necessary.**
+
+## This library's API is huge!
+
+**The W3C does not require client applications to support all of these
+use cases.** The W3C only requires that *"all implementations must at least be
+capable of serializing and deserializing the Extended properties in accordance
+with the Activity Streams 2.0 Core Syntax,"* which what this library and the
+`activity/vocab` libraries do for clients. This library's API is large to
+permit clients to use as much or as little as desired.
+
+## What it does
+
+This library provides a `Resolver`, which is simply a collection of callbacks
+that clients can specify to handle specific ActivtyStream data types. The
+`Resolver.Deserialize` method turns a JSON-decoded `map[string]interface{}`
+into its proper type, passed to the corresponding callback.
+
+For example, given the data:
+
+```
+{
+  "@context": "https://www.w3.org/ns/activitystreams",
+  "type": "Note",
+  "name": "Equivalent Exchange",
+  "content": "I'll give half of my life to you and you give half of yours to me!",
+  "attachment": "https://example.com/attachment"
+}
+```
+
+in `b []byte` one can do the following:
+
+```
+var m map[string]interface{}
+if err := json.Unmarshal(b, &m); err != nil {
+	return err
+}
+r := &Resolver {
+	NoteCallback: func(n *Note) error {
+		// 1) Use the Note concrete type here
+		// 2) Errors are propagated transparently
+	},
+}
+if handled, err := r.Deserialize(m); err != nil {
+	// 3) Any errors from #2 can be handled, or the payload is an unknown type.
+	return err
+} else if !handled {
+	// 4) The callback to handle the concrete type was not set.
+}
+```
+
+Only set the callbacks that are interesting. There is no need to set every
+callback, unless your application requires it.
+
+## Using concrete types
+
+The convenience layer provides easy access to properties with specific types.
+However, because ActivityStreams is fundamentally built off of JSON-LD and
+still permits large degree of freedom when it comes to obtaining a concrete type
+for a property, the convenience API is built to give clients the freedom to
+choose how best to federate.
+
+For every type in this package (except `Resolver`), there is an equivalent type
+in the `activity/vocab` package. It takes only a call to `Raw` to go from this
+convenience API to the full API:
+
+```
+r := &Resolver {
+	NoteCallback: func(n *Note) error {
+		// Raw is available for all ActivityStream types
+		vocabNote := n.Raw()
+	},
+}
+```
+
+To determine whether the call to `Raw` is needed, the "get" and "has" methods
+use `Resolution` and `Presence` types to inform client code. The client is free
+to support as many types as is feasible within the specific application.
+
+Reusing the `Note` example above that has an `attachment`, the following is 
+client code that tries to handle every possible type that `attachment` can
+take. **The W3C does not require client applications to support all of these
+use cases.** 
+
+```
+r := &Resolver {}
+r.NoteCallback = func(n *Note) error {
+	if n.LenAttachment() == 1 {
+		if presence := n.HasAttachment(0); p == ConvenientPresence {
+			// A new or existing Resolver can be used. This is the convenient getter.
+			if resolution, err := n.ResolveAttachment(r, 0); err != nil {
+				return err
+			} else if resolution == RawResolutionNeeded {
+				vocabNote := n.Raw()
+				// Use the full API
+				if vocabNote.IsAttachmentIRI(0) {
+					...
+				} else ...
+			}
+		} else if p == RawPresence {
+			vocabNote := n.Raw()
+			// Use the full API
+			if vocabNote.IsAttachmentIRI(0) {
+				...
+			} else ...
+		}
+	}
+}
+```
+
+## Serializing data
+
+Creating a raw type and serializing it is straightforward:
+
+```
+n := &Note{}
+n.AddName("I'll see you again")
+n.AddContent("You don't have to be alone when I leave")
+// The "type" property is automatically handled...
+m, err := n.Serialize()
+if err != nil {
+	return err
+}
+// ...but "@context" is not.
+m["@context"] = "https://www.w3.org/ns/activitystreams"
+b, err := json.Marshal(m)
+```
+
+The only caveat is that clients must set `"@context"` manually at this time.
+
+## What it doesn't do
+
+This library does not use the `reflect` package at all. It prioritizes
+minimizing dependencies and speed over binary size.
+
+The ActivityStream specification is built on top of JSON-LD, which uses JSON.
+This library should be used with `encoding/json` in order to transform a raw
+string into a `map[string]interface{}` to static, semantically meaningful
+types.
+
+This library does not set the `"@context"` property required when sending
+serialized data. Clients are in charge of setting it to
+`"https://www.w3.org/ns/activitystreams"`.
+
+This implementation is heavily opinionated against understanding JSON-LD due to
+its sacrifice of semantic meaning, significant increase of complexity, even
+weaker typing, and increased exposure to partially-understood messages. These
+costs earn a degree of flexibility that is not needed for the ActivityStream
+Vocabulary.
+
+This library is *not* a [JSON-LD](https://json-ld.org/) parser, and by design
+does not implement any further understanding of JSON-LD that may be outlined in
+the [W3C's JSON-LD](https://www.w3.org/TR/json-ld/) specification. Furthermore,
+it does *not* implement any of the JSON-LD
+[processing algorithms](https://www.w3.org/TR/json-ld-api/). If this
+functionality is strictly needed, or this library is not suitable, please see 
+[piprate/json-gold/ld](https://github.com/piprate/json-gold) and its
+[documentation](https://godoc.org/github.com/piprate/json-gold/ld).
+
+## Other considerations
+
+This library is entirely code-generated. Determined clients can add their own
+custom extended types to the `activity/tools/defs` library and generate a
+useful type. However, this process is purposefully painful to force clients to
+seriously consider whether they need their own
+[custom type](https://xkcd.com/927).
+
+The code-generation aspect also allows the specification to be translated into
+declarative data, which permits certain kinds of validation and verification.
+This has led to giving the following feedback to the specification:
+
+* [Inconsistencies between property domains and type properties](https://github.com/w3c/activitystreams/issues/436)
+* [Please clarify "items" and "orderedItems" properties](https://github.com/w3c/activitystreams/issues/437)
+* [Expectations around IRI resolution](https://github.com/w3c/activitystreams/issues/438)
+* [Examples Contradict Specification](https://github.com/w3c/activitystreams/issues/439)
+* [Tombstone "formerType" Property Range Needs Clarification](https://github.com/w3c/activitystreams/issues/440)
+* [Example 60 Missing `@context` Property](https://github.com/w3c/activitystreams/issues/441)
+* [Stylistic Consistency For Non-Functional Single-Value JSON in Examples](https://github.com/w3c/activitystreams/issues/442)
+* [Spec does not clarify non-functional natural language values when mapped](https://github.com/w3c/activitystreams/issues/443)
+* [Example 102: `url` property missing `type: "Link"`](https://github.com/w3c/activitystreams/issues/444)
+* [Example 146: Missing `Z` in startTime](https://github.com/w3c/activitystreams/issues/445)
+* [Example 150: Latitude/Longitude are not xsd:float](https://github.com/w3c/activitystreams/issues/446)

streams/activitystreams_test.go

@@ -0,0 +1,228 @@
+//go:generate go install github.com/go-fed/activity/tools/streams
+//go:generate streams
+package streams
+
+import (
+	"encoding/json"
+	"github.com/go-fed/activity/vocab"
+	"github.com/go-test/deep"
+	"testing"
+)
+
+func TestRepoExamples(t *testing.T) {
+	for name, ex := range allRepoExamples {
+		resFn := func(s vocab.Serializer) error {
+			m, err := s.Serialize()
+			if err != nil {
+				return err
+			}
+			m["@context"] = "http://www.w3.org/ns/activitystreams"
+			actual, err := json.Marshal(m)
+			if diff, err := GetJSONDiff(actual, []byte(ex)); err == nil && diff != nil {
+				t.Errorf("%s: Serialize JSON equality is false:\n%s", name, diff)
+			} else if err != nil {
+				t.Errorf("%s: GetJSONDiff returned error: %s", name, err)
+			}
+			return nil
+		}
+		r := &Resolver{
+			ObjectCallback: func(x *Object) error {
+				return resFn(x)
+			},
+			LinkCallback: func(x *Link) error {
+				return resFn(x)
+			},
+			ActivityCallback: func(x *Activity) error {
+				return resFn(x)
+			},
+			IntransitiveActivityCallback: func(x *IntransitiveActivity) error {
+				return resFn(x)
+			},
+			CollectionCallback: func(x *Collection) error {
+				return resFn(x)
+			},
+			OrderedCollectionCallback: func(x *OrderedCollection) error {
+				return resFn(x)
+			},
+			CollectionPageCallback: func(x *CollectionPage) error {
+				return resFn(x)
+			},
+			OrderedCollectionPageCallback: func(x *OrderedCollectionPage) error {
+				return resFn(x)
+			},
+			AcceptCallback: func(x *Accept) error {
+				return resFn(x)
+			},
+			TentativeAcceptCallback: func(x *TentativeAccept) error {
+				return resFn(x)
+			},
+			AddCallback: func(x *Add) error {
+				return resFn(x)
+			},
+			ArriveCallback: func(x *Arrive) error {
+				return resFn(x)
+			},
+			CreateCallback: func(x *Create) error {
+				return resFn(x)
+			},
+			DeleteCallback: func(x *Delete) error {
+				return resFn(x)
+			},
+			FollowCallback: func(x *Follow) error {
+				return resFn(x)
+			},
+			IgnoreCallback: func(x *Ignore) error {
+				return resFn(x)
+			},
+			JoinCallback: func(x *Join) error {
+				return resFn(x)
+			},
+			LeaveCallback: func(x *Leave) error {
+				return resFn(x)
+			},
+			LikeCallback: func(x *Like) error {
+				return resFn(x)
+			},
+			OfferCallback: func(x *Offer) error {
+				return resFn(x)
+			},
+			InviteCallback: func(x *Invite) error {
+				return resFn(x)
+			},
+			RejectCallback: func(x *Reject) error {
+				return resFn(x)
+			},
+			TentativeRejectCallback: func(x *TentativeReject) error {
+				return resFn(x)
+			},
+			RemoveCallback: func(x *Remove) error {
+				return resFn(x)
+			},
+			UndoCallback: func(x *Undo) error {
+				return resFn(x)
+			},
+			UpdateCallback: func(x *Update) error {
+				return resFn(x)
+			},
+			ViewCallback: func(x *View) error {
+				return resFn(x)
+			},
+			ListenCallback: func(x *Listen) error {
+				return resFn(x)
+			},
+			ReadCallback: func(x *Read) error {
+				return resFn(x)
+			},
+			MoveCallback: func(x *Move) error {
+				return resFn(x)
+			},
+			TravelCallback: func(x *Travel) error {
+				return resFn(x)
+			},
+			AnnounceCallback: func(x *Announce) error {
+				return resFn(x)
+			},
+			BlockCallback: func(x *Block) error {
+				return resFn(x)
+			},
+			FlagCallback: func(x *Flag) error {
+				return resFn(x)
+			},
+			DislikeCallback: func(x *Dislike) error {
+				return resFn(x)
+			},
+			QuestionCallback: func(x *Question) error {
+				return resFn(x)
+			},
+			ApplicationCallback: func(x *Application) error {
+				return resFn(x)
+			},
+			GroupCallback: func(x *Group) error {
+				return resFn(x)
+			},
+			OrganizationCallback: func(x *Organization) error {
+				return resFn(x)
+			},
+			PersonCallback: func(x *Person) error {
+				return resFn(x)
+			},
+			ServiceCallback: func(x *Service) error {
+				return resFn(x)
+			},
+			RelationshipCallback: func(x *Relationship) error {
+				return resFn(x)
+			},
+			ArticleCallback: func(x *Article) error {
+				return resFn(x)
+			},
+			DocumentCallback: func(x *Document) error {
+				return resFn(x)
+			},
+			AudioCallback: func(x *Audio) error {
+				return resFn(x)
+			},
+			ImageCallback: func(x *Image) error {
+				return resFn(x)
+			},
+			VideoCallback: func(x *Video) error {
+				return resFn(x)
+			},
+			NoteCallback: func(x *Note) error {
+				return resFn(x)
+			},
+			PageCallback: func(x *Page) error {
+				return resFn(x)
+			},
+			EventCallback: func(x *Event) error {
+				return resFn(x)
+			},
+			PlaceCallback: func(x *Place) error {
+				return resFn(x)
+			},
+			ProfileCallback: func(x *Profile) error {
+				return resFn(x)
+			},
+			TombstoneCallback: func(x *Tombstone) error {
+				return resFn(x)
+			},
+			MentionCallback: func(x *Mention) error {
+				return resFn(x)
+			},
+		}
+		m := make(map[string]interface{})
+		err := json.Unmarshal([]byte(ex), &m)
+		if err != nil {
+			t.Errorf("%s: Cannot json.Unmarshal: %s", name, err)
+			continue
+		}
+		// Examples that needed adjustment:
+		// Example 64: Array of one attachment - deserializes OK, but re-serialization does not match
+		// Example 68: Array of one bcc - deserializes OK, but re-serialization does not match
+		// Example 70: Array of one cc - deserializes OK, but re-serialization does not match
+		// Example 112: Array of one item - deserializes OK, but re-serialization does not match
+		// Example 118: Array of one tag - deserializes OK, but re-serialization does not match
+		// Example 123: Array of one to - deserializes OK, but re-serialization does not match
+		// Example 184: missing @context
+		// Example 196: '\n' characters were literal newlines in source
+		err = r.Deserialize(m)
+		if err != nil {
+			t.Errorf("%s: Cannot Resolver.Deserialize: %s", name, err)
+			continue
+		}
+	}
+}
+
+func GetJSONDiff(str1, str2 []byte) ([]string, error) {
+	var i1 interface{}
+	var i2 interface{}
+
+	err := json.Unmarshal(str1, &i1)
+	if err != nil {
+		return nil, err
+	}
+	err = json.Unmarshal(str2, &i2)
+	if err != nil {
+		return nil, err
+	}
+	return deep.Equal(i1, i2), nil
+}

tools/streams/main.go

@@ -0,0 +1,19 @@
+package main
+
+import (
+	"github.com/go-fed/activity/tools/defs"
+	"github.com/go-fed/activity/tools/streams/gen"
+	"io/ioutil"
+)
+
+func main() {
+	allTypes := append(defs.AllCoreTypes, defs.AllExtendedTypes...)
+	b, err := gen.GenerateConvenienceTypes(allTypes)
+	if err != nil {
+		panic(err)
+	}
+	err = ioutil.WriteFile("activitystreams.go", b, 0666)
+	if err != nil {
+		panic(err)
+	}
+}

vocab/README.md

@@ -8,11 +8,11 @@ specification linked above in addition to usual unit tests.
 Its mission is simple: Provide meaningful static types for the ActivityStream
 Vocabulary in golang.
 
-This library is entirely code-generated by the `activity/vocab/gen` library
+This library is entirely code-generated by the `tools/vocab/gen` library
 and `tools/vocab` tool. Run `go generate` to refresh the library, which
 which requires `$GOPATH/bin` to be on your `$PATH`.
 
-**Please consider using the `activity/stream` library instead.**
+**Please consider using the `activity/streams` library instead.**
 
 ## This library's API is huge!
 
@@ -20,7 +20,7 @@ which requires `$GOPATH/bin` to be on your `$PATH`.
 use cases.** The W3C only requires that *"all implementations must at least be
 capable of serializing and deserializing the Extended properties in accordance
 with the Activity Streams 2.0 Core Syntax,"* which what this library and the
-`activity/stream` libraries do for clients. This library's API is large to 
+`activity/streams` libraries do for clients. This library's API is large to 
 permit clients to use as much or as little as desired.
 
 ## What it does
@@ -70,7 +70,7 @@ func (n *Note) GetNameString(index int) string { ... }
 
 Note that the resulting API and property type possibilities is *large*. This is
 a natural consequence of the specification being built on top of JSON-LD. It is
-recommended for applications to use the `activity/stream` convenience library
+recommended for applications to use the `activity/streams` convenience library
 instead, or create their own convenience types:
 
 ```
@@ -113,7 +113,7 @@ functionality is strictly needed, or this library is not suitable, please see
 ## Other considerations
 
 This library is entirely code-generated. Determined clients can add their own
-custom extended types to the `tools/vocab` tool and generate a useful type.
+custom extended types to the `tools/defs` library and generate a useful type.
 However, this process is purposefully painful to force clients to seriously
 consider whether they need their own [custom type](https://xkcd.com/927).
 
@@ -136,4 +136,4 @@ This has led to giving the following feedback to the specification:
 ## Thanks
 
 Many thanks to those who have worked on JSON-LD, ActivityStreams Core, and the
-ActivityStreams Vocabulary.
+ActivityStreams Vocabulary specifications.