Skip to content

Issue: Broadcaster does not receive all listeners consistently in Conference mode (participant visibility inconsistency) Description #333

Description

We are facing an inconsistency in ConnectyCube VideoChat Conference (SFU) where participant visibility behaves differently for broadcaster and listeners.

Expected Behavior

Broadcaster View

Broadcaster should be able to see all listeners who have joined the conference.

Even if a listener has camera/microphone off, they should still appear as a participant tile (at least as an audio-only or placeholder tile).

Listener View

Listener should be able to see:

The main broadcaster

Other listeners who are part of the conference

In short:

All participants should see a consistent participant list across roles (broadcaster & listener).

Actual Behavior

Broadcaster only sees participants when onRemoteStreamListener fires.

If a listener:

joins with camera off

or joins as a listener-only role

or publishes no media stream

→ the broadcaster never receives that participant and cannot render a tile.

Listener views behave differently and appear more consistent.

This creates:

Missing participant tiles

Role-based inconsistencies

UI desync between broadcaster and listeners

here is the relavent code which i have used :

join meeting handler

` const joinMeeting = (
userName,
roomId,
userId,
camClass,
isVideo,
isAudio,
asListener
) => {
return new Promise((resolve, reject) => {
createCallbacks();
participantRef.current[0].userId = userId;
setParticipants([...participantRef.current]);
console.log("inside join participants", participants);

  ConnectyCube.videochatconference
    .getMediaDevices()
    .then((allDevices) => {
      let mediaParams = mediaDevices(allDevices);
      if (!mediaParams.audio && !mediaParams.video) {
        reject("Error:You do not have any camera and microphone available");
        return;
      }
      const session = ConnectyCube.videochatconference.createNewSession();
      _session.current = session;

      setDevicesStatus({ video: isVideo, audio: isAudio });

      // Listener path: do NOT acquire local media, join directly
      if (asListener) {
        participantRef.current[0].isListener = true;
        participantRef.current[0].isMicroMuted = true;
        participantRef.current[0].isVideo = false;
        participantRef.current[0].stream = null;
        participantRef.current[0].isListener = true;
        withVideo.current = false;
        isVideoMutedRef.current = true;
        setIsVideoMuted(true);
        setParticipants([...participantRef.current]);
        return session
          .joinAsListener(roomId, userId, userName)
          .then(() => {
            setIsLoaded(true);
            const devs = mediaDevices(allDevices);
            setDevices(devs);
            resolve(devs);

            return null;
          })
          .catch((err) => {
            console.log(err);
            reject(err);
            return null;
          });
      }

      // Publisher path: acquire camera/mic
      const devicesVisible = { video: isVideo, audio: isAudio };
      withVideo.current = true;
      isVideoMutedRef.current = !isVideo;
      setIsVideoMuted(!isVideo);

      session
        .getUserMedia(devicesVisible)
        .then((localStream) => {
          // Respect initial camera choice by disabling the track
          try {
            if (!isVideo) {
              const v =
                localStream.getVideoTracks &&
                localStream.getVideoTracks()[0];
              if (v) v.enabled = false;
            }
          } catch (e) {}

          participantRef.current.filter((p) => p.name === "me")[0].stream =
            localStream;

          const newParticipants = [...participantRef.current];
          setParticipants(newParticipants);
          ConnectyCube.videochatconference
            .getMediaDevices(
              ConnectyCube.videochatconference.DeviceInputType.VIDEO
            )
            .then((videoDevices) => {
              setChoosedCam(videoDevices[1]?.deviceId);
              setCams(videoDevices);
            })
            .catch((error) => {
              console.log(error);
            });
          // asListener handled earlier; do nothing here

          session
            .join(roomId, userId, userName)
            .then(() => {
              setIsLoaded(true);
              setDevices(mediaParams);
              const msg = {
                body: isVideo ? "camera__on" : "camera__off",
                extension: {
                  photo_uid: "7cafb6030d3e4348ba49cab24c0cf10800",
                  name: "Our photos",
                },
              };

              ConnectyCube.chat.sendSystemMessage(userId, msg);
              resolve(mediaParams);
            })
            .catch((error) => {
              console.log(error);
              alert(JSON.stringify(error));
              reject(error);
            });
        })
        .catch((error) => {
          console.warn(
            "[joinMeeting] getUserMedia(video=true) failed, falling back to audio-only + dummy video",
            error
          );
          if (!_session.current || !_session.current.getUserMedia) {
            reject(error);
            return;
          }
        });
    })
    .catch((error) => {
      console.log(error);
      reject(error);
    });
});

};`

Create callbacks Handlers
` const createCallbacks = () => {

console.log('insideonParticipantJoinedListener')
ConnectyCube.videochatconference.onParticipantJoinedListener = (
  session,
  userId,
  userDisplayName,
  isExistingParticipant
) => {
  activeUsersRef.current.add(userId);
  let user = participantRef.current.find(
    (participant) => participant.userId === userId
  );
  console.log('user =============>', user)
  if (!user) {
    participantRef.current.push({
      userId,
      name: userDisplayName,
      stream: null,
      connectionStatus: "good",
    });
  } else {
    participantRef.current.map((obj, id) => {
      if (obj.userId === userId) {
        obj.name = userDisplayName;
      }
      return obj;
    });
    const newParticipants = [...participantRef.current];
    setParticipants(newParticipants);
  }
  const isVideoMuted = {
    body: isVideoMutedRef.current ? "camera__off" : "camera__on",
    extension: {
      photo_uid: "7cafb6030d3e4348ba49cab24c0cf10800",
      name: "Our photos",
    },
  };
  const microStatus = {
    body: participantRef.current[0].isMicroMuted
      ? "micro__muted"
      : "micro__unmuted",
    extension: {
      photo_uid: "7cafb6030d3e4348ba49cab24c0cf10800",
      name: "Our photos",
    },
  };
  const sharingStatus = {
    body: participantRef.current[0].isSharing ? "sharing" : "not-sharing",
    extension: {
      photo_uid: "7cafb6030d3e4348ba49cab24c0cf10800",
      name: "Our photos",
    },
  };

  const listenerStatus = {
    body: participantRef.current[0].isListener
      ? "is-listener"
      : "is-not-a-listener",
    extension: {
      photo_uid: "7cafb6030d3e4348ba49cab24c0cf10800",
      name: "Our photos",
    },
  };

  ConnectyCube.chat.sendSystemMessage(userId, sharingStatus);
  ConnectyCube.chat.sendSystemMessage(userId, microStatus);
  ConnectyCube.chat.sendSystemMessage(userId, isVideoMuted);
  ConnectyCube.chat.sendSystemMessage(userId, listenerStatus);
  const newParticipants = [...participantRef.current];
  setParticipants(newParticipants);
};

ConnectyCube.videochatconference.onParticipantLeftListener = (
  session,
  userId
) => {
  console.log("======> participant leaving", userId);

  activeUsersRef.current.delete(userId);

  // stop remote stream
  const p = participantRef.current.find(e => e.userId === userId);
  if (p?.stream) {
    p.stream.getTracks().forEach(t => t.stop());
  }

  participantRef.current = participantRef.current.filter(
    (e) => e.userId !== userId
  );
  const newParticipants = [...participantRef.current];
  setParticipants(newParticipants);
  console.table(participantRef.current);
};
ConnectyCube.videochatconference.onRemoteStreamListener = (
  session,
  userId,
  stream
) => {
  try {
    console.log("[onRemoteStreamListener] userId=", userId, stream);
  } catch (e) {}
  participantRef.current.map((obj, id) => {
    if (obj.userId === userId) {
      obj.stream = stream;
      obj.isVideo = stream ? true : false;
    }
    return obj;
  });
  const newParticipants = [...participantRef.current];
  setParticipants(newParticipants);
};
ConnectyCube.videochatconference.onSlowLinkListener = (
  session,
  userId,
  uplink,
  nacks
) => {
  participantRef.current.filter((e) => {
    if (e.userId === userId) {
      e.connectionStatus = "average";
    }
    return e.connectionStatus;
  });
  setParticipants([...participantRef.current]);
  let existingSlowLinkTimer = slowLinkTimersRef.current[userId];
  if (existingSlowLinkTimer) {
    clearTimeout(existingSlowLinkTimer);
  }
  slowLinkTimersRef.current[userId] = setTimeout(() => {
    participantRef.current.filter((e) => {
      if (e.userId === userId) {
        e.connectionStatus = "good";
      }
      return e.connectionStatus;
    });
    setParticipants([...participantRef.current]);
  }, 4000);
};
ConnectyCube.videochatconference.onRemoteConnectionStateChangedListener = (
  session,
  userId,
  iceState
) => {

};
ConnectyCube.videochatconference.onSessionConnectionStateChangedListener = (
session,
iceState
) => {
};

navigator?.mediaDevices.addEventListener("devicechange", function (event) {
  setDevices({ video: true, audio: true });
});

ConnectyCube.chat.onSystemMessageListener = (msg) => {


  let idx = participantRef.current.findIndex(
    (participant) => participant.userId === msg.userId
  );


  if (msg.extension.sharing === "sharing" || msg.body === "sharing") {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isSharing = true;
  } else if (
    msg.extension.sharing === "not-sharing" ||
    msg.body === "not-sharing"
  ) {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isSharing = false;
  }
  // for listener
  if (msg.body === "is-listener" || msg.body === "is-listener") {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isListener = true;
  } else if (
    msg.body === "is-not-a-listener" ||
    msg.body === "is-not-a-listener"
  ) {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isListener = false;
  }

  if (msg.body === "camera__off") {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isVideo = false;
    setParticipants([...participantRef.current]);
  } else if (msg.body === "camera__on") {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isVideo = true;
    setParticipants([...participantRef.current]);
  }
  if (msg.body === "micro__muted") {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isMicroMuted = true;
    setParticipants([...participantRef.current]);
  } else if (msg.body === "micro__unmuted") {
    participantRef.current.find(
      (p) => p.userId === msg.userId
    ).isMicroMuted = false;
    setParticipants([...participantRef.current]);
  }
};

};`

Please let me know how can i make this scenerio possible in my webApp ????

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions