Skip to content

Instantly share code, notes, and snippets.

@Jerdak
Last active February 13, 2022 16:32
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Jerdak/87ecbe4bcb0ba9e33b10 to your computer and use it in GitHub Desktop.
Save Jerdak/87ecbe4bcb0ba9e33b10 to your computer and use it in GitHub Desktop.
3-Dimensional (3D) file format based on Google's protobuf API.
#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import print_function
import mesh_pb2 as pobj
def dump_mesh(mesh):
""" Dump mesh data up to, at most, 10 vertices
and 10 faces.
The 10 vert/face limit is simply to limit
the amount of data on screen.
"""
print("Mesh name: " + mesh.name)
print("Num. Verts: %d"%(len(mesh.vertices)))
print("Num. Faces: %d"%(len(mesh.faces)))
for index,vertex in enumerate(mesh.vertices):
print("V[%d]: %f %f %f"%(index,vertex.x,vertex.y,vertex.z))
if index == 10:
break
for index,face in enumerate(mesh.faces):
print("f[%d count: %d]: %d %d %d"%(index,face.poly_type,face.vert_index[0],face.vert_index[1],face.vert_index[2]))
if index == 10:
break
def test_protobuf_read():
""" Test protocol buffer reader
"""
with open("tmp.pb","rb") as fr:
mesh = pobj.Mesh()
mesh.ParseFromString(fr.read())
dump_mesh(mesh)
def test_protobuf_write():
""" Test protocol buffer write
Assumes you have a valid `load_mesh`
function.
"""
with open("tmp.pb","wb") as fw:
# load vertex and face data however you like
verts,faces = load_mesh("test.ply")
mesh = pobj.Mesh()
mesh.name = "protobj_py_test"
for v in verts:
#protobuf repeated message fields are RepeatedMessageFields and are created with add()
mv = mesh.vertices.add()
mv.x = v[0]
mv.y = v[1]
mv.z = v[2]
for f in faces:
mf = mesh.faces.add()
# protobuf repeated non-message fields are treated as RepeatedScalarFieldContainers
# so use 'append', not 'add'
mf.vert_index.append(f[0])
mf.vert_index.append(f[1])
mf.vert_index.append(f[2])
test_protobuf_write()
test_protobuf_read()
package protobj;
// 4D Vertex
message Vertex {
optional float x = 1 [default=0];
optional float y = 2 [default=0];
optional float z = 3 [default=0];
optional float w = 4 [default=1];
}
// RGB color (each color in [0,255])
message Color {
optional uint32 r = 1 [default=255];
optional uint32 g = 2 [default=255];
optional uint32 b = 3 [default=255];
}
// Texture coordinate
message UV {
optional float u = 1 [default=0];
optional float v = 2 [default=0];
optional float w = 3 [default=1];
}
// Polygonal face
message Face {
optional uint32 poly_type = 1 [default=3]; //polygonal type = number of sides
repeated uint32 vert_index = 2 [packed=true]; //vertex array index
repeated uint32 uv_index = 3 [packed=true]; //uv array index
repeated uint32 norm_index = 4 [packed=true]; //normal array index
repeated uint32 order = 5 [packed=true]; //Original order (to preserve original array order)
}
// Mesh data
message Mesh {
optional string name = 1;
optional string comments = 2;
// Core
repeated Vertex vertices = 3; //Vertex array, Should *never* be empty
repeated Face faces = 4; //Face array, can be empty
repeated Vertex normals = 5; //Normal array, can be empty
repeated Color colors = 6; //Color array, can be empty
repeated UV uvs = 7; //UV array, can be empty
// Extension support
extensions 2000 to max;
}

##Protocol mesh object

A 3-Dimensional (3D) mesh file format based on Google's protobuf library.

Intent

Create a more robust and extensible 3D file format that is similar to Stanford PLY but less cumbersome to extend and easily adaptable to multiple languages like C++, Python, and Java. The format should be similar in flexibility to JSON or XML but compact (binary) and much faster.

There is nothing inherently wrong with Stanford PLY and it's certainly well supported by most commercial applications. The issue is that not all implementations of the PLY library are consistent because, frankly, it's old and maintained by fringe graphics users. Google's protocol buffers are a future-proof way of efficiently encoding structured data in a way that is obvious to most engineers.

Protobuf rules

  • required field specifiers are not allowed. As the docs say, "Required is forever". Use optional or repeated instead.
  • repeated fields should note whether they should have 0 elements or not. protobuf allows repeated fields to have any number of elements, including 0. However for certain fields you may wish to stipulate that if they are 0 any implementation should flag this as an error. This is a social agreement, protobuf has not mechanism to control this behavior.
  • Extensions must be supported and should start at a sufficiently high index to avoid future collisions
  • optional fields must explicitly define a default value. protobuf will revert to a type-specific value if default is not specified.
  • repeated fields of basic numeric types will include [packed=true] to provide more efficient packing when available.

Content rules

  • Do not normalize data if it isn't necessary. e.g. Store RGB colors as int32s, not floats. This allows protobuf's optimizer to use variable length encoding for a smaller footprint.
  • The core .proto file should never be changed once established. protobuf extensions will provide support for future enhancements but that means the core must have group consensus.
  • Export tools must exist before the format is distributed. This includes standalone tools and plugins for Blender, MeshLab, Maya, and 3DS Max.
@davidB
Copy link

davidB commented Feb 6, 2015

Hi, I'm also working on a 3D format using protobuff (I just found your page). If
want to take a look the current name is pgex (can change this month).
I also work on a Blender exporter (part of blender_external_renderer ) and a jMonkeyEngine importer (part of jme3_ext_remote_editor)

Any feeback is welcome, maybe I'll try your store of RGB.
Can I link your page ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment