Log in

No account? Create an account
Conlang Evaluation - Constructed Languages

> Recent Entries
> Archive
> Friends
> Profile
> Language Creation Society

Language Creation Society
Language Construction Kit
Lexical Semantics of a Machine Translation Interlingua
On the Design of an Ideal Language

December 16th, 2006

Previous Entry Share Next Entry
03:51 am - Conlang Evaluation

Conlang evaluation: a framework for criticism
By Sai Emrys - conlangs@saizai.comhttp://saizai.com
16 December 2006


David Peterson posted an entry in January 2002 later entitled the Conlanger’s Manifesto, an apologia in defense of conlanging as an art to be taken seriously [1]. In March 2002, Jesse Bangs wrote an entry [2] on the CONLANG mailing list entitled “Lighting Some Flames: Towards conlang artistry”, and later known as the Artlanger’s Rant. It went one step further, advocating for a new body of conlang criticism. This part was relatively well received, except that he also advocated for “schools” of conlanging – his being the Naturalist school – and that triggered a rather… heated debate.

Most of the problems people had with the idea of conlanging schools came under two basic categories:
1. Concern that doing so would split up what is already a relatively small subculture (approximately 1000 members worldwide, of whom probably 100 are publicly active), and set the schools in opposition to each other.
2. Belief that conlang criticism could not be done because the intention of the creator can vary so widely – and that, e.g., applying Jesse’s Naturalist criteria to languages that don’t hold the same values as his would be inapplicable at best or ideologically arrogant at worst.

When I posted questions asking about readers’ *own* priorities in prep for this essay, the response was by contrast completely positive and non-flamewar-like. Perhaps this is from the community having learned from the previous experience (and avoiding those subjects), but I believe it is in what exactly is asked, and what the assumptions behind the questions are.


My intent here is not to reignite that 2002 controversy, but to offer an alternative – one that serves the purposes of Jesse Bangs in giving a framework for and encouraging substantive conlang critique; is value-neutral and takes into account the varying goals of language creators; and is fun and easy to use yet comprehensive.

In my opinion – and from my surveys – there is no doubt that there are numerous criteria that can be applied to examine a given language (whether natural or created). For example, one can ask how well documented it is; how closely its features fit in the bell-curve of natural language ‘universals’; how aesthetically pleasing it is from certain perspectives; how regular its grammar is; how easy it is to learn.

Some of these questions can be evaluated objectively. To rate ‘universality’ for example (see below), take each of its features (e.g. split ergativity) for which there is a known occurrence frequency in natural languages, and compute an average number using those frequencies to say “how normal is this language, compared to all known natural languages”.

Others can only be evaluated subjectively. For example, phonaesthetics will vary widely depending on the reader’s personal preferences, cultural background, etc. At best for this one can try to define some types of aesthetics that are relatively consistent and agreeable (such as “mellifluousness”), and rate the language for that rather than for general subjective “aesthetic” in the “sounds good” or “sounds bad” sense – because tastes can vary so widely.

I must emphasize strongly that none of these criteria have any *inherent* value to them. Some creators may prefer to have a highly regular language; others may prefer one that is just irregular enough to add ‘spice’; still others may actually prefer a language that is so irregular as to be cryptographically baffling.

As such, though each criterion can be evaluated, there is no inherent “value” rating to the criteria – i.e. no concept of “good” or “bad”, merely specific value-neutral qualitative questions. They should be treated solely at face value for each item.


The criteria I describe here can be used as a sort of Conlang Code, à la Geek Code et al. To distinguish from the Conlang*er* Code v1.1, I dub this the Conlang Code v2. They can be used in parallel and mostly do not overlap. [3]

For each element, ratings are listed. They can be used alone for classification and rating purposes, but for a real review it would of course be more helpful to include a per-criterion breakdown where relevant explaining the reasons for evaluating each item as such, or with suggestions for how to improve it (toward the author’s desired state). They can also be used to state what your goals are for your language(s).

Each criterion is rated on a scale from ---- to ++++, with some other options (using !) for when the answer falls outside that spectrum. Alternatively, criteria can be listed in numeric format for concision. E.g. nat-(+++) = nat-1(3).

Variables work much like the Conlanger Code:
! placed after: not enough data to evaluate (review)
* placed after: item does not apply (review); indifference, neutrality (goals)
[item omitted]: item was not evaluated (review); indifference / neutrality (goals)
() placed after: value varies (e.g. foo++(+++) = foo varies from ++ to +++)
! placed before: tertium quid or other special variation (see descriptions)

Code abbreviations are the capital letters of their heading. (E.g. nat for NATuralness). They are listed in lowercase (“nat”) in the code or as a rating (“nat++”), but in uppercase when referring to the criterion itself (“NAT is a criterion that….”).

Some items have sub-criteria listed under ‘extras’. These are optional.

Most of these criteria can be approached both objectively and subjectively. In a detailed review, one should expect to find for each point the rating followed by specific reasons (possibly empirical ones, possibly not) why that rating was given.

All reviews are *only* expected to be accurate of what is publicly documented as of the date of the review.

The review should normally be of the *primary* mode of the language. In most cases, this will be the spoken one. (See MOD.)


Part 1: CLCv2 (for “ConLang Code version 2” – this part is invariant)

Part 2: Date of writing this code (mandatory!), in DMonYY format

Part 3: Code usage type
R Conlang Review (by author or others)
G Conlang Goals (permissible by author only)

Part 4: Language(s) covered by this code (author may have separate Goal codes for separate languages, or cover multiple ones in one code if they’re similar enough in goals)

Part 5: Priority (goals version only)
++++ I work on this nearly all the time.
++ This is my main current project.
-- Enh, someday I’ll edit it.
---- No longer under development and not likely to be any time soon.

If priority differs, list appended to each language name; otherwise just append to the last one.

E.g. a review might start with: CLCv2 6Dec06 R Foolang …
Or a goals code might start with: CLCv2 23May05 G Erisian-, Discordian++ …


In deference to Jesse, I will start with his "naturalist" school's primary criteria.

NATuralness: Appears to be a normal human Earth language.
nat++++ At the apex of normality: exactly 2.3 children, half a divorce, etc. (Within 0.5 s.d. of mean.)
nat++  Medium normal. No linguist would blink an eye at finding a description of it in Languages of the World. (Within 2 s.d.s of mean.)
nat  Within the bounds of normal, but perhaps somewhat quirky. (Within 3 s.d.s of mean.)
nat--  Not really normal at all, but maybe if you squint *really* hard… (Within 4 s.d.s of mean.)
nat----  Just plain bizarre. 10 clicks, split ergativity, and a built-in encryption algorithm? (What mean?)
!nat++  This language is fairly normal… for the world/culture in which it’s set (“Well, if there *were* a blind space faring avian race, this probably *would* be a pretty normal language for their physiology…)

E.g. languages: French: nat++ Esperanto: nat; All Noun: nat---
Goals: Jesse Bangs: nat++(+++); ye generic loglanger: nat*; ye generic sci-fi author: !nat+

ComPLeteness: Can be used to talk about anything.
cpl++++ Shakespeare, astrophysicists, and politicians can all use it.
cpl+++  Can be used to translate Hamlet.
cpl+  Can be used to translate the Babel Text
cpl  Enough for tourists to get directions…
cpl--  Well, I’m kinda still working on orthography…
cpl----  I only just named it!
!cpl++  It can be used to talk about most subjects… within the limited scope it’s intended for. (e.g. “Battle-Command-ese”)

E.g. languages: English cpl++++; Toki Pona cpl+(+++); Quenya: cpl+; ye generic toylang: cpl—
Goals: ye generic auxlanger: cpl++++; ye generic sci-fi author: cpl

ComPleXity: Number of rules, exceptions to those rules, and various interacting contradictory levels of grammar.
cpx++++ Might require a doctorate and supercomputer to be able to use in real time.
cpx++  Wait, I thought you said I comes before E and sounds like /i/…
cpx  At least it’s no harder than my Spanish class in high school.
cpx--  So there’re a couple small exceptions, but otherwise it’s pretty simple and regular…
cpx----  There’s only a dozen rules? No exceptions? Everything else is free?

E.g. languages: Ithkuil cpx++++; Sanskrit cpx+(+++); Toki Pona cpx----
Goals: ye generic auxlanger: cpx--(---); ye generic artlanger: cpx+(++); ye generic loglanger: cpx*

Note: complexity of rules does not necessarily equate to complexity of output. Viz. John Conway’s /Life/, or the Turing machine.

Personal INnovation: Breaks away from the author’s preconceptions about how language works (i.e. their L1s).
pin++++ You, a non-linguistically-trained native English-only speaker,  came up with split ergativity by yourself?
pin++  Playing with bits not present in your L1 language family…
pin----  Another relex, eh.
!pin  Sure it’s a euroclone and the author’s L1s are English and French, but zie is 100% aware of all the places zie copied zir native grammar and explicitly decided to do so.
!!pin  The author seems to be rejecting their L1 grammar without real thought!

E.g. languages: almost everybody’s first conlang (be honest now): pin---
Goals: my students’ for their final project: pin++

Global INnovation: Something never done before in human language.
gin++++ [insert doctoral thesis and Nobel science award here]
gin++   You have a language… with no nouns!?
gin  The language has a unique “feel”.
gin----  Yes, people have made European auxiliary languages before…
!gin+  That’s a pretty unusual language concept… for a sketch / fiction story.

E.g. languages: ye generic IAL: gin----; Heptapod B: !gin+++
Goals: ye generic loglanger: gin+++(++++); ye generic artlanger: gin*()

CoHeRence: Overall fit, consistency of ‘feel’, etc.
chr++++ Every singly aspect of the language is brilliantly tailored to the same goal.
chr++  Everything fits together well, with a very definite “feel”.
chr--  Somebody’s been learning new things in Linguistics class again…
chr----  Design by committee using dice rolls, a random word generator, and every grammatical feature ever mentioned on CONLANG.

E.g. languages: Toki Pona chr+++; my conlang class languages: chr--(---)
Goals: ye generic loglanger, artlanger, and auxlanger: chr++++

CuLTural expressiveness: Shows that it is attached to a specific culture.
clt++++ A sociolinguist could create a complete analysis of the mores, technological level, and societal structure of the culture that uses this language just from looking at the language.
clt++  The language is clearly tied to a strong cultural
clt----  You can’t even tell that such a thing as sociolinguistics exists judging from this language.

E.g. languages: Quenya clt++
Goals: ye generic auxlanger: clt----; ye generic artlang: clt++++

LIBeralness of culture expressed: In the generic American sense.
lib++++ We love everyone! No biases! Completely gender-equal! Sex positive!
lib----  Only the righteous deserve respect.
!lib  Does not sufficiently express a culture in the first place.

E.g. languages: Laadan lib++; Newspeak lib---
Goals: ye generic auxlanger: !lib; ye generic feminist conlanger: lib+++

MeLliFluousness: Smooth, pretty, poetic in its phonaesthetic.
mlf++++ The most vile curses sound like lullabies.
mlf----  Spoken by an angry Klingon with a bad cold.

E.g. languages: Quenya mlf++(++++), Klingon mlf---(----)

SaPir-Whorf: Encourages (or forces) a certain worldview through its structure.
spw+++++ Grants new powers – genius, psychic ability, etc. – or permanently changes the speaker’s psychology.
spw++++ Makes it impossible to even think about something against its design.
spw++  Strongly encourages a specific way of looking at the world.
spw----  Exactly the same as speaking any other generic L1.
!spw+++ Does not *require* it, but does *allow* one to think about things nearly inexpressible or inconceivable in a generic L1.

E.g. languages: Heptapod B spw+++++; Toki Pona spw+++(++++); Laadan !spw++
Goals: ye generic cognitive psycholinguist: spw+++;

EASe of learning: Ease with which this language would be learned (as an L2)
eas++++ Anybody could be fluent within days.
eas----  Would require a PhD and 20 years of exclusive study just to ask offhand how to find the bathroom.
!eas++[eu] Qutie easy to learn… if you already know a eurolang. (See below for the two-letter abbreviations.)

Extras: L1 acquisition ease (format: eas[L2 ease]:[L1 ease])

E.g. languages: ye generic African click language eas---:+; Toki Pona eas++; Ithkuil eas---
Goals: ye generic auxlanger eas+++(++++); ye generic euro-auxlanger !eas+++(++++)[eu]; ye generic a priori loglanger eas*:++

DOCumentation: Thoroughness of the *publicly available* documentation.
doc++++ Dictionaries, linguistic analysis, primers – the works. And all easily understandable.
doc++  Thorough description available, enough to learn the language if you look.
doc  Enough to at least get a pretty good feel for what it’s all about
doc--  More of a sketch than an actual documentation. Missing some important bits.
doc----  It’s all in the author’s head… or so zie claims.

Extras: list 2-letter codes in which the documentation is available. E.g. doc++[en,sp,fr]
Extras v2: for each code or codes, separately rate thoroughness. E.g. doc---[en]:+++[jp]

If not specified in extras, the default is English.

E.g. languages: Laadan doc++++; NLF2DWS doc--
Goals: ye generic hobbyist: doc*; ye generic cryptolinguist: doc----; ye generic auxlanger: doc++++

CoRPus: Variety and volume of in-language material available.
crp++++ Thousands of novels, audio/video recordings, poetry, scientific documents, etc.
crp  There’s a Babel text…
crp----  You mean I’m supposed to *use* this language?

E.g. languages: English crp++++; ye generic sketch conlang crp--
Goals: ye generic loglanger crp*; ye generic auxlanger crp++++

FINishedness: Stablity & completion of the grammar and vocabulary.
fin++++ Essentially not changing at all.
fin+++  Not changing *intentionally*, but is changing through usual natlang social mechanisms.
fin++  Quasi stable. Still in progress but no major revisions expected.
fin----  Might completely change without warning tomorrow.

E.g. languages: NLF2DWS fin--; Lojban fin++(++++)
Goals: ye generic artlanger fin*; ye generic auxlanger fin++++

FIDelity: How much the map looks like the territory.
fid++++ Everything possible is a near perfect *matopaeic match.
fid++  Most things are at least “translucent”, i.e. they make sense as representations once you know what they mean.
fid----  Everything is completely arbitrary. You would never guess what something means without being told.

E.g. languages: ASL: fid++(+++)
Goals: ye generic non-engelanger: fid*; NLF2DWS: fid++(+++)

EFFort: How hard it is to produce / understand sentences (includes both physical and mental effort).  Also includes length of time required.
eff++++ Takes hours to figure out how to say it, and then leaves you exhausted to actually try.
eff----  Very easy and fast to speak and comprehend.

Extra: Separate difficulty from time. (Format: eff[difficulty]:[time])

E.g. languages: Speedwords eff--:----; Ithkuil eff+++(++++)
Goals: ye generic auxlang: eff----; ye generic artlang or engelang: eff*

DeNSity: How much information is transmitted in a certain amount of space (or time).
dns++++ Could write the whole plot of Hamlet on one piece of paper.
dns++  One page of English text translates to 20 seconds of speech.
dns----  One page of English text translates to *20* pages…

E.g. languages: Ithkuil dns++; formal Japanese dns--
Goals: ye generic engelang: dns++(++++); ye generic artlang: dns*

CLaRity: Ease of saying exactly what one wants, no more and no less
clr++++ Can be both completely ambiguous about whatever one wants and retain absolute precision about the rest.
clr  About the normal amount of ambiguity or forced specification for natlangs.
clr----  Forced to over-specify or under-specify greatly.

Extra: Distinguish “allowed” from “forced” clarity. (Format: clr[allowed]:[forced] E.g. clr+++:--- = can be very specific, but language requires specifying e.g. everybody’s gender, sexual orientation, and hair color when referring to them.)

E.g. languages: Lojban clr++; ye generic gender-neutral artlang clr:++ (= clr0:2)
Goals: ye generic loglang: clr++++; ye generic artlang clr*

NoiSE resistance: How well utterances are understood when in noisy situations (shouting, bombs, static, bad TV reception, speech impediments, etc)
nse++++ You could use it in the middle of a heavy artillery barrage and still be able to discuss the finer points of Kant as applied to astrophysics without being misunderstood.
nse  Fairly hard to get the message right if there’s a lot of noise; might be misunderstood if there’s a mild amount.
nse----  Anything less than perfect clarity means you’re unlikely to get the message even close to correct.

Extra: Semantic difference type, aka the Hungarian phrasbook index (Format: nse[resistance]:[diff type]
nse:++++ Small difference in form à vastly different or completely unrelated meaning (e.g. mother à Chomskian meta-pragmatics or mother à “I cannot permit your ugly face to continue existing, prepare to die”). Aka “tourists likely to get killed”.
nse:----   Small difference in form à similar or less specific meaning (e.g. mother à brother or mother à relative). Aka “graceful degradation”.

E.g. languages: ye generic triconsonantal-root language (for vowels only): nse:--(---)
Goals: ye generic engelang: nse++++:----

Form / Concept Complexity matching: How much complex ideas have complex forms, and simple ideas simple forms.
fcc++++ “Now” = 1 phone. Postnuclearantidisestablishmentarianism = 20.
fcc  About even distribution
fcc----  Postnuclearantidisestablishmentarianism = 1 phone. “Male” =  20.
!fcc  Random relationship

FAMily: Extent to which this conlang is part of a greater family of languages
fam++++ Part of an extended family of languages, with multiple intersecting lines, intermixing, protolanguages, etc.
fam  Part of one line of languages (e.g. “Old Conlish” vs “New Conlish” vs “modern slang Conlish”)
fam----  What family? This language stands on its own.
!fam++ This con-creole is plausible… if Malaysia ever *did* invade Morocco... (Rating = how well it fits in its adopted real-world-natlang family.)

MODalities: Diversity and robustness of language’s modes
mod++++ Can be spoken, written, signed, or maybe even smelt; utilizes each mode’s medium uniquely; each mode conveys full spectrum of meaning (e.g. emotion, undertones, etc)
mod  Has a unique writing system
mod---- Only exists in one mode.

Note: Most conlangs are oral with incidental written codes. Some add a more robust writing system, e.g. ideographic. More extensively developed languages may require a separate evaluation for each mode (i.e. treat the oral, sign, and written languages as separate languages in the same family).

DireCTness: How straightforward the language’s idioms are
dct++++ Wanna fuck?
dct  Some of both…
dct----  I see a book I might read.  Do you have it? *
!dct  The language doesn’t have idioms as such, because it isn’t strongly tied to any particular culture.

* from Sally Caves’ Teonaht (“Kyam rykken ellepma wemry hain. Ain fy dhar ha?”)

TiLT: Overall rating of all factors not covered above (Reviews only)
tlt++++ This is the best language I have ever seen.
tlt++  Unusually good.
tlt   About average.
tlt----  This is the worst language I have ever seen.

Extra: Separately specify tilt for aesthetics vs technique. (Format: tlt[aesthetics]:[technique].)

E.g. tlt++ = excellent overall. tlt++:- = very pretty, but not particularly technically adept.
Goals: ye generic artlang: tlt+++:*; ye generic engelang: tlt*:+++


AMBition of design goals: Difficulty of implementing the author’s design goals
amb++++ Might well require a lifetime’s hard work by a brilliant mind.
amb  Would require at least a few months to accomplish.
amb---- A newbie could do that over a weekend.

Note: the amb score should take into account the number of design goals listed, how difficult each one would be, how difficult it would be to fulfill competing goals, etc.

E.g. languages: ??
Goals: ye brilliant artlang: amb++(+++); English relex: amb----

SUCcess: Success at fulfilling the stated goals of the language. (Reviews only)
suc++++ Scores 0 = perfect.
suc  Scores 5 = average / needs work.
suc----  Scores 9 = managed to do the exact opposite of what was intended.
!suc  Goals are not available.

To calculate the SUC score, for each item – except AMB – for which there is a goal rating available (i.e. not * or unlisted), take the absolute value of the goal rating minus the review rating. Average all these numbers and round down.

E.g. goals: eff4 mlf4 cpx-2 (= eff++++ mlf++++ cpx--)
Review: eff4 mlf-1 cpx0 (= eff++++ mlf- cpx)
SUC: avg(abs(4-4), abs(4-[-1]), abs(-2-0)) = avg(0, 5, 2) = 7/3 = 2.3 = suc++


In all of the foregoing, I have tried to be meticulous in emphasizing that these criteria are completely value-neutral… of themselves. No criterion is inherently more or less ‘important’ or ‘valuable’ than any other.

However, it is nonetheless true that people both have goals for their own languages, and beliefs or values as to what makes for a better or worse language (or at least, one that they personally like more or less). This is reflected in using this framework to declare one’s goals.

I believe the most relevant evaluation, then, is one where many people rate a language on the specific criteria above, and this is compared against the author’s goals to determine how well they met those goals. This score is admittedly subjective in many of its parts, but at least cannot be criticized as applying an inapplicable standard, as it is the author zirself who decides what criteria are important.

This also allows for an easy way to define conlang “schools” as fuzzy categories. Using various statistical and neural net methods – with people desiring similar things generally grouping together – one can distinguish certain broad or specific categories of goal-sets. Keeping with fuzzy categories avoids the too-rigid and perhaps polemic concept of fixed ‘schools’ as such. However, one would expect to see a grouping of e.g. artlangers and loglangers separate out if you only look for two groups, as the two tend to value very different things.

An “ideal” or truly “successful” language is, thus, one that reaches all its design goals perfectly.

Some design goals will be more ambitious than others of course, as reflected in AMB. I think we can agree that a language that is highly successful *and* ambitious is more deserving of praise than one that is only one or the other.


I believe it to be undeniably obvious that conlanging is a *craft*, like being an architect of a living system. All architects must incorporate some degree of both aesthetics and technical mastery, but which aspects are more relevant will depend greatly on the project.

1. Figure out what you are really aiming for – both in the context of the ideas mentioned here, and ones not covered – and publish your goals along with your language. This framework should help, as a starting point, to clarify for yourself what sort of language you are trying to create. Invite people to review your language.

2. Review both your own conlangs and others’. Use this framework as a beginning from which to review a language – to give the review from structure and “get to the point” with ratings. Supplement it with more detailed descriptions for those points that are interesting, need work, or are done particularly well.

Please note, this system intentionally does *not* cover linguistic typology or linguistic features. There are simply far too many to cover; doing so would require another, rather more cumbersome, system. More importantly, though, linguistic features are not *inherently* purposeful, and only indirectly related to the ‘feel’ of a language or to whether one is likely to think more highly of it. This system is meant to rate those more core, meta elements of how a language is perceived.

I have tried to ensure that this system is maximally comprehensive for 95% of people, and has a minimal amount of overlap between its criteria. It is not intended to be able to be 100% comprehensive; that is not possible in any finite rating system. It should however cover the goals of all types of languages; most languages will probably find many of these ratings irrelevant, and that is fine. The point is, different languages will find different things to be relevant.

However, I may have overlooked something; if so, please let me know, and we can make version 2.1. ;-)


- Sai

[1] http://archives.conlang.info/ce/jusae/theilvepian.html
[2] http://archives.conlang.info/fo/jinbua/vhavheintian.html
[1, 2] these entries are archived, with retrospective annotation by Jesse Bangs, here: http://web.archive.org/web/20041022065450/http://students.washington.edu/jaspax/conlang.htm
[3] http://taliesin.nvg.org/conlang/conlang-code-1.1.html

This can be used in conjunction with the conlang code v1.1 above. To do so, use the format:
[conlang code v1.1 or later minus ‘name of conlang’ item]
CLCv2 [date] G language(s)[priority] [insert CLCv2 code  here] [linebreak]
G language(s)[priority] [insert CLCv2 code here] [etc as relevant]

(15 comments | Leave a comment)


Date:December 16th, 2006 05:01 pm (UTC)
I rather like this system. Well, besides the use of the (+) and (-) ranking, but that's just me being picky about unfamiliar information design. Still, do you think you’d be able to find individuals both willing and qualified to perform such an evaluation?

Also, it should be noted that the goals of conlangs do change over time. To perform such an evaluation on anything but a “finished” language might not be terribly relevant; at best it might give an author an idea of where they’re currently going with the project. I think the usefulness of such a critique would have to depend on how likely it is to encourage authors to revise their language to better meet the specified criteria they’ve set forth. On the other hand, I’m sure that for a lot of people the “completion” of their conlang marks a point of complete satisfaction with the project so that whatever the evaluation comes up with will automatically represent their perfect ideal. This is unless perhaps they go about making demonstrably false claims about their language (Say, it’s easy to learn when it’s got more complex system of inflections than Ancient Greek)

For myself, I feel that I’m the only one qualified to judge how well my conlang meets my own expectations. But then, perhaps that’s because the most important design specs I’m working with are more qualitative and personal than anything quantitative and measurable. Say, how well the basic vocabulary and its organization into a semantic hierarchy fits my own unique outlook and implicit sense of reason. How well the logograms suit my taste. Other qualities like the ease of learning the spoke language, or the ease of writing the logograms are also important, but more or less secondary.
[User Picture]
Date:December 17th, 2006 06:46 am (UTC)
All conlangs are generally works in progress. IMO that should not prevent critique any more than it does for, e.g., architecture design. All good projects have ongoing feedback-and-improvement cycles.

You're only the only one qualified to judge if it meets your expectations IF you don't really explain what they are. Once you do, it's fair game for others to comment on (beyond "it looks pretty to seo_cwen, which is just another version of "you haven't really explained what you want"). Qualitative elements included - we do judge art after all, and most real elements (e.g. how well the semantic hierarchy fits [outlook defined here]) *can* be addressed by others.
Date:December 17th, 2006 04:53 pm (UTC)
With most visual arts you'd never want to show off an unfinished work for critism (Unless you're a high school student) because what really matters is the unity of the final product; presenting an unfinished work will only give your audience a false impression of it. It's fine for getting non-artists to ooh and aw at your sketchbook doodles while bored to death in a Roman History lecture, but it's really not part of the professional process. Unless the artist is dead, generally you’re only going to find their finished pieces hung up in a gallery. Likewise, even if a poet or writer has an editor anyone worth their meat isn’t going to use one for anything more than hunting down rogue typos.

Architecture (and any other field of design rather than fine art) is different because the artist in question usually needs to meet the approval of a client before they can expect millions of dollars to be dumped into throwing the building up.

Furthermore, do you think any critic or curator in any other field of art or design would ever use an evaluation sheet like this? If you’re in school, if you’re lucky, then maybe; in my experience, teachers are more likely to ignore the rubric, give you a letter grade and maybe a paragraph of overly vague comments. I mean, you’re trying to turn art criticism into an exercise in statistics which it most certainly is not. In fact, usually the judgment of art really is subjective.

I mean, the world of fine arts today is little more than an economic bubble blown out of soapy politics by pack of elitist snobs who never had the talent to pursue art themselves; you’re no likely to hear any real patron of the arts justify themselves with any more than, “This piece speaks to me,” Not that things were any better when artists had to live off the dole of wealthy patrons.

I do think there’s such a thing as “good” and “bad” art, but then, you’re never going to win a debate over what makes a piece good or bad. It’s worse than arguing religion. Nor do I think how well the artist meets their original goals is a very good answer; Livy wanted the Aeneid burned after all and for most of Western History it’s been regarded as the defining poem of Western Civilization.
[User Picture]
Date:December 20th, 2006 05:37 pm (UTC)
I mean, you’re trying to turn art criticism into an exercise in statistics which it most certainly is not.

I second this. Besides, I think feedback from others is useful when it includes qualified discussion of particular details; just digits without verbose comments don't tell much - e. g. about the criticist's vision of how a specific criterion is applied.

Cf. my comment below: without knowing that -10 naturalness refers specifically to ten clicks plus complex sandhi with a given reviewer, the message is completely meaningless for me. And I'm afraid that would be the default case with bare codes.
[User Picture]
Date:December 17th, 2006 06:47 am (UTC)
Oh, and the +/- system is taken simply because it's the de facto standard for codes. Viz. Geek Code, Conlanger Code, et al.
[User Picture]
Date:December 16th, 2006 05:52 pm (UTC)
I think the past means of evaluation of conlangs has served well, which is rating them on the author's goals/priorities. This comes from, at least, modern expectations of art. Rothko was critiqued on representing the human form. Picasso wasn't critiqued (successfully) on photo-realist goals. Mostly, as in art, those goals are self-evident when we look at a particular conlang.

If someone is focusing on respresenting dense meaning, it looks very different from a conlang where the alien species

I like the idea of a conlang code, but I don't want it to be a review (reviews, since they're semi-subjective, and discussable are best kept to a natural common language--e.g. English). I think we should try to keep separate your codes into the
1. empirical properties (that the conlanger will self-report; e.g. documentation available)
2. goals of the conlanger (that they are holding up as the criteria they're most interested in being evaluated by)
3. subjective review conclusions that should only be done by a reviewer (e.g. your innovation and coherence variables) which might also work in English reviews.

So then the protocol, would be that in the conlanger's posts (and maybe in their profile, .sig, etc), they give their conlang code which includes 1+2, and maybe people post reviews in (3). I would maintain, if it's an art (and most of us agree that it is) the strict rubric of (3) would only be necessary in conlang slams or competitions or something. Reviews in other mediums (magazines, etc) are half-oriented towards helping us _appreciate_ the beauty once we look at the subject of the review--not to distill it to a teacher's grade.
[User Picture]
Date:December 16th, 2006 05:53 pm (UTC)
sorry, that's "Rothko was NOT critiqued on the human form" :-/
[User Picture]
Date:December 17th, 2006 06:54 am (UTC)
You *can* critique Rothko on form, or Picasso on photorealism. It's simply not necessarily what they are aiming for. That's why SUC is there. That doesn't mean though that it's useless to discuss how photorealistic Picasso is.

Conlanger self-report - actually IMO they're usually not qualified to judge the quality of their own documentation, any more than you would trust a coder to write technical and user manuals without oversight by someone whose job that is. They simply don't have the perspective of an outsider learning / learning about the language.

Goals and review use (almost) the exact same full code. I don't see how this is something that could be separated out (it just doesn't make sense), so it seems to me that at least one of us is misunderstanding the other.

Personal innovation - again, that can be a goal. Not everyone CARES whether they are breaking away from their L1 assumptions. Whether you are competent to review yourself on that point is just a subcomponent of the same question for everything (viz. documentation above). You can certainly try though.

Yes, reviews IMO should be geared towards either communicating appreciation (if written for an audience) or improving the product (if written for the conlaner). Having a structured way to review things helps for both.
(Deleted comment)
[User Picture]
Date:December 20th, 2006 06:37 am (UTC)
Speaking as moderator rather than as the author of this post:

Do not troll. If you have criticism you are always welcome to share it, but it must be constructive rather than simply rude.

This is a formal warning. If you do it again, you will be banned for a month.
[User Picture]
Date:December 20th, 2006 07:16 am (UTC)
Geekcode-type profiles have traditionally* been used as a method of self-description, not outside evaluation.

[User Picture]
Date:December 20th, 2006 07:25 am (UTC)
And this scan be used as both.
[User Picture]
Date:December 20th, 2006 04:57 pm (UTC)
I'm not sure I understand NAT. For example, this line:

nat---- Just plain bizarre. 10 clicks, split ergativity, and a built-in encryption algorithm? (What mean?)

I thought I knew what was meant by "natural" at least by people toying with naturalistic conlangs. I thought it was something like "resembling a human natlang". But then I don't understand four dashes for "10 clicks" (while natlangs that actually use clicks may have 70+) and, say, some Sanskrit-style sandhi rules ("built-in encryption", aren't they, essentially?).

It appears that NAT was mixed with (reverse) GIN, in *examples*.

Well, I believe I could formulate a couple less obvious bits on naturalness, but I'm afraid they might seem idiosyncratic and controvrsial, including to some "naturalist school" people. OTOH perhaps they'd provoke some useful discussion. What do you think?
[User Picture]
Date:December 20th, 2006 11:00 pm (UTC)
Clicks are rare. Split ergativity is rare. Neither alone is *that* unusual; together they're still within the range of normal.

But by encryption I meant the real sort - i.e. one where you would need to know a shared secret to be able to understand the message. Sandhi are not nearly that bad. ;-)

Encryption in the first place is non-natural (AFAIK no natural langauge has this feature); together with other natural but rare features, it's simply not natural at all.

Keep in mind, NAT is not meant to be "good/bad" or boolean - only to say that there *is* a range of naturalness (read: "normal for a natural language") even within natural languages themselves. Natlangs do vary, and some are very odd - as it were, on the fringes of the bell curve, but within the scope of 'natural' because they have occured through natural mechanism (i.e. definitionally). Conlangs vary even more, and may well fall outside the spectrum of what is plausibly a 'natural language'.
[User Picture]
Date:December 21st, 2006 03:31 pm (UTC)
My point wasn't about good/bad. I meant, simply, that naturalness and typological (un)commonness are two different things. The former translates as "looking like a natlang". The latter is about typologically rare features. Auxlangs often avoid rare features, and this is part of the reason why they don't look natural. If you think the position on the bell curve (which I'd like to see yet, though) is important, such a criterion should be given a different name. I am definitely in the "naturalist" zone of conlanger diversity spectrum, and I definitely don't accept NAT measured as commonness. I can try to explain how I would measure it and it would be nice to see the feedback from other people in the same "school", since I realize very well that I can't speak for others.

Now, just commenting...

Split ergativity is rare.

Depends on how you define it. With most definitions I've seen, every language where the repertory of tenses available for passive is different from that used in active must be counted as split-ergative. And even if you solve this, it's clear that the spectrum is sort of continuous.

Sandhi are not nearly that bad.

OK, you don't mean one can parse Sanskrit sentences in real-time mode without knowing the key (actual sandhi rules), do you? ;)

AFAIK no natural langauge has this feature

There are "natural" secret languages, sometimes with rather formal encryption patterns, and communities where everybody including children can speak and understand the encoded version of the local dialect as well as its mixture with the normal version (e. g. roots encoded, suffixes not). Such communities existed in Russia back in 1970'ies, for example. So I'm not sure about "not natural at all".

Conlangs vary even more [...]

Only in some aspects, you know. And some (e. g. lexification strategies) are overly monotonous, when compared to natlangs (IMHO).

[User Picture]
Date:July 7th, 2011 07:53 pm (UTC)
I liked this article on conlang evaluation so much I rewrote a condensed outline of it without the geek code on my blog.


I'd make the comment that it omits real-life communities as a dimension for judging a language.

Also, a good number of these criteria would require some level of competency in the target language to apply-- the person who is smart enough to gain significant competency in a conlang is smart enough to be even handed and fair-- (maybe). Criteria that can be applied when speed reading the typical scant material available online would be more practical.

> Go to Top