CMM - map validation service

For all your great and interesting ideas
bludshot
Posts: 111
Joined: Fri Sep 23, 2011 12:12 am

Re: CMM - stamp of quality for maps

Post by bludshot »

I think you should reconsider this idea. I'm not saying you shouldn't do it, but you should do it wisely and carefully.

First of all, many of the things you listed to be checked should honestly be done in a computer program, not by humans. You could (find someone to) write a computer program similar to a HTML validator, an UrT map validator, that would check all the shaders against some master list, check all the graphics for powers of 2, and check all the other stuff you talked about checking.

There is a disconnect between what you say this stamp of quality is for and what people are going to perceive it is for. When I first saw the title and top of this thread I assumed it was CMM's stamp of approval for the *quality* of the maps (obviously, since that's what it says), but then you go on to describe a process that doesn't consider the quality of the map, but merely technical issues.

Sure, technical issues do fall under the umbrella of quality, but quality is much more broad than that. And to the average gamer, the technical stuff hardly even matters while the true quality of the map does.

What is the point of a CMM stamp of quality if terrible garbage maps would qualify for the stamp? It would become completely meaningless to the end user, and put CMM in a bad light. End users will not say "Oh wow, this map has only textures with a power of 2, great!", they will say "Wow this map is shit, this CMM thing must also be shit" (in the case of a shitty map that meets the standards for the CMM stamp, which so far are exceedingly low standards from what I'm reading. Pretty much all you need to do is not have errors or technical flaws in the map. So a giant box map with cubes in it would get the CMM stamp. A poorly scaled map with an ugly theme would get the stamp, etc.

Personally, I think if you want to do this thing as described, then you should look into my map validator idea, and drop the stamp but instead just have a validator that mappers can quietly use without any obvious CMM stamps imho.

If you want the CMM stamp thing, imo you need way better requirements, but, I think then you may find yourself in a quandary about what is a good map and what isn't. I think it would be quite difficult.

User avatar
xandaxs
Posts: 959
Joined: Wed Aug 17, 2011 5:22 pm
Location: Oeiras, Portugal

Re: CMM - stamp of quality for maps

Post by xandaxs »

Well... I believe this is just a scratch, and we'll come up with some sort of minimum quality required...

As you may have seen, we usually give feedback on layout changes/improvements aswell as textures, report bigs and so on.. That would all be taken in consideration imo
[12:25] <JohnnyEnglish> morning Nounou
[12:25] <JohnnyEnglish> wotcha doing?
[12:25] <Nounou> hello
[12:26] <Nounou> nothing much, i've nothing to do at work so
[12:26] <Nounou> modeling woman
[12:26] <JohnnyEnglish> woo
[12:26] <JohnnyEnglish> real women?
[12:26] <Nounou> realistic yes, on maya

Image

User avatar
johnnyenglish
Space cake
Posts: 898
Joined: Wed Aug 17, 2011 9:23 am
Location: Nottingham, England

Re: CMM - stamp of quality for maps

Post by johnnyenglish »

Bludshot, that is good advice and the title of this topic was badly chosen.

Your map validator is something I'd like to hear more about and I'd gladly install such a device on this forum so that mappers could upload their pk3 and the validator would display any problems.

I am also suggesting a technical review of a map by testers on clean installs of 4.1, 4.11 and urtalpha prior to release to raise the quality and to prevent 99% of common problems before the map reaches the servers.

Once a map has the reputation of being "buggy" because it has a missing texture or sound it's difficult to remove that perception.

I never intended to suggest that the test should offer advice about layout/style - those are subjective and depend on personal tastes.

I guess I'll leave the offer open and if anyone decides to submit a map for testing we'll devise the process at that time and develop it as we go along.

I agree this should be a discrete certification and not include anything that suggests that CMM thinks that the map is totally groovy, just that it was tested and found to work.

bludshot
Posts: 111
Joined: Fri Sep 23, 2011 12:12 am

Re: CMM - stamp of quality for maps

Post by bludshot »

Oh ok. I mean when I saw those seals I was getting the sense that you meant like slapping those on the loading screens or something lol

User avatar
Rayne
Site Admin
Posts: 624
Joined: Wed Aug 17, 2011 5:22 pm

Re: CMM - stamp of quality for maps

Post by Rayne »

Like Bludshot i too was under the impression that the overall quality of the map would be put to a test.
I should have read all the posts more carefully.
What you guys are talking about is basically beta testing. Checking the map for errors before it reaches the servers.
For this purpose making some custom stamp vector thingy is ridiculous, you can't slap stamps on every new beta a mapper produces.
Those things go on final releases, and when you review an RC version you don't bother players with the technical side.
Like Blud said none of them gives a damn if textures a power of 2. All they care about is if the map is worth playing.
Gameplay
Appearance
Performance
Call it "C.M.M.'s GAP test" or something. Focus on 3 most important aspects and don't go into technical details.
If you read a game review you'll notice that nobody writes about how the engine produces HDR lighting. They just say "lighting in the game is fucking awesome" and that's all a player wants to hear. Is it good or is it bad.

bludshot
Posts: 111
Joined: Fri Sep 23, 2011 12:12 am

Re: CMM - stamp of quality for maps

Post by bludshot »

Rayne, Johnny is talking about a technical 'map validator' to check that there are no obvious technical errors.

So the idea of a "gameplay - appearance - performance" certification would be a separate one. My opinion is that a certification like that would be a lot of work, as well as very subjective, and I'm not really sure it would be productive.

I mean, I know how to do a stamp like that, all of us could write up a list of requirements and then you could have a committee that checks them and votes on the map meeting the standard or whatever... But, I honestly don't see what the point of it is. To me, something like that would be the job of the game devs if they wanted to clearly let the public know which maps (or servers) are running good quality 'real' urban terror and which are lame-o maps that don't represent UrT properly. But FS has never shown any interest in doing that, too much work and/or they don't care.

User avatar
Rayne
Site Admin
Posts: 624
Joined: Wed Aug 17, 2011 5:22 pm

Re: CMM - stamp of quality for maps

Post by Rayne »

I know what he's talking about, and once again that's called beta testing. If you can't find and fix obvious errors during beta tests then you shouldn't be a mapper.
I don't see why would FS team be the one that determines which maps are good and which aren't. It's same like saying FS team made the game they should run all the leagues, or that they should be the ones that run UAA.
You can make maps for left 4 dead as well but i don't see their dev team testing 3'rd party maps. It makes no sense to me.

User avatar
xandaxs
Posts: 959
Joined: Wed Aug 17, 2011 5:22 pm
Location: Oeiras, Portugal

Re: CMM - stamp of quality for maps

Post by xandaxs »

Whatever it is that you choose to do, I want to have the map Toxic tested by you guys after ai update it
[12:25] <JohnnyEnglish> morning Nounou
[12:25] <JohnnyEnglish> wotcha doing?
[12:25] <Nounou> hello
[12:26] <Nounou> nothing much, i've nothing to do at work so
[12:26] <Nounou> modeling woman
[12:26] <JohnnyEnglish> woo
[12:26] <JohnnyEnglish> real women?
[12:26] <Nounou> realistic yes, on maya

Image

User avatar
johnnyenglish
Space cake
Posts: 898
Joined: Wed Aug 17, 2011 9:23 am
Location: Nottingham, England

Re: CMM - stamp of quality for maps

Post by johnnyenglish »

Rayne wrote:I know what he's talking about, and once again that's called beta testing. If you can't find and fix obvious errors during beta tests then you shouldn't be a mapper.


We've all (most of us) released a map, texture pack or model with minor errors, duh mistakes or just forgotten something. For example I once forgot to include an arena file, I've released maps with 4096x4096 images and missing shaders and a whole load of other errors.

An example :

Bobby worked very hard to produce ireland b4 which was derived from ireland b2 and had been tested extensively but still had some minor issues -especially when played without 4.1.1. Not everyone installs 4.1.1, not all servers run it.

Had Bobby submitted his map to this process we'd have spotted the missing textures, the shader problems and the model problems before he released it.

Mappers aren't the best validators of their own work, they generally have all the assets somewhere on their system and perhaps only one version of urt installed.

As I've said, if you want to submit your map to the process just before you release - we'll check it over, if any problems are found you'll be notified in private.

Having it checked over before it hits the servers may save some tears/red face later and by not releasing buggy maps into the community it will raise the standard and opinions of 3rd party maps.

Oh, and If you don't want to submit your map - don't. :D

Post Reply